Google's Project Ellmann: AI-Powered 'Bird's-Eye' View of Users' Lives Raises Privacy Concerns
Google is exploring the use of artificial intelligence (AI) technology, named "Project Ellmann," to create a comprehensive "bird's-eye" view of users' lives by analysing mobile phone data, including photographs and searches. According to a presentation that CNBC watched, the project, which makes use of biographer Richard David Ellmann, aims to use large language models (LLMs) like Gemini to find patterns in user photos, create a chatbot, and answer questions that were previously unanswerable.
While it remains unclear if these capabilities will be integrated into Google Photos or another product, Project Ellmann's proposal coincides with Google's recent launch of its advanced AI model, Gemini. Gemini, touted as Google's most capable AI model to date, outperforms OpenAI's GPT-4 and is designed to process information beyond text, including images, video, and audio.
A product manager for Google Photos presented Project Ellmann alongside Gemini teams at an internal summit. The teams concluded that large language models are ideal for creating a holistic life story. Using biographies, previous moments, and contextual information, Ellmann aims to describe a user's photos more profoundly than merely labelling pixels and metadata.
The presentation highlighted the capability to identify life "moments" such as university years, Bay Area residence, and parenting experiences, emphasising the need for a "bird's-eye view" to answer complex questions and tell compelling stories. Project Ellmann could infer significant events, like a user's child's birth, by utilising knowledge from various points in the user's life.
In addition to the life story aspect, the presentation introduced "Ellmann Chat," a feature that envisions a ChatGPT-like interface already knowledgeable about the user's life. Users could inquire about their pet's siblings' visits or seek recommendations for similar towns for potential relocation.
The technology demonstrated its ability to summarise a user's eating habits, identify considered purchases, discern interests, work, and travel plans from screenshots, and determine favourite websites and apps.
Google responded to the revelations, stating that the concept is in the early exploration stage. The company emphasised its commitment to responsible development, prioritising user privacy.
Project Ellmann represents Google's bid to compete among tech giants to create more personalised life memories, surpassing the existing features in Google Photos and Apple Photos. However, concerns about user privacy and the responsible use of personal data will likely be central to the public's response to such ambitious AI-driven initiatives.