Project Astra is “She” of Google. And they want it to work with augmented reality glasses

The company revealed how it sees the future of assistants with artificial intelligence

The bet is similar to OpenAI’s, but with a shiny new twist: Google plans to integrate it into glasses.

The avalanche of news about Google’s generative AI models has culminated with a special announcement. Project Astra is Google’s ambitious initiative to create the voice assistant of the future. One is based on artificial intelligence and vision provided by the cameras of our smartphones (or, as we will see, other devices).

Project Astra. This is the name of this ambitious project by Google, with which they want to “develop universal AI agents that can be useful in our everyday lives.” Google emphasized that an assistant with these characteristics should understand and respond in the same way as humans. Also, “remember what you see and hear so you can understand the context and act.”

Latency is a problem. Those responsible for the company acknowledge that they have made great progress in understanding multimodal information (that is, text, voice, audio and video data). However, “improving response time on something conversation-oriented is a difficult engineering challenge.”

Voice tones. Project Astra is working to offer voice synthesis models that are higher quality and allow different agents to have a greater variety of intonations. According to Google, agents will be able to understand the context in which they are used better than ever and will also respond quickly.

Gemini app is on the horizon. All of this training will eventually be integrated into solutions like the Gemini mobile app, which will be equivalent to what OpenAI introduced with the GPT-4o-based ChatGPT we saw yesterday. The application, Google indicates, will be available before the end of the year.

Tell me what you see. A demo video released by Google shows a preview version of Project Astra in action, with mobile cameras performing amazing object recognition. In fact, Google came up with interesting options, such as drawing an arrow on the screen and then asking the model to describe what the object on the screen was. This reminds us of the new Search Circle feature, but in this case applies to those live queries about things that the AI ​​model recognizes.

Glasses! The most powerful part of the video came in the final part, when the person doing the demonstration asks, “Where did I leave my glasses?” The assistant tells him, and when he puts them on, it turns out that these are not ordinary glasses: they are glasses with a camera and Project Astra integration! (Some, by the way, remind you of something). Next, the user briefly demonstrates how, through this integration, the model helps and answers the user’s questions in a wonderful and inventive way.

OpenAI is ahead, but Google is lagging behind. Thus, Project Astra appears to be a direct competitor to the features that OpenAI presented yesterday in GPT-4o. The Sam Altman-led firm appears to have made some headway as these voice interaction options are already slowly reaching some users, but their widespread availability will likely take a few weeks… or months. Google is a little behind, but this alternative seems just as promising and would be an interesting way to stimulate this particular competition. We, the users, will win, that’s for sure.

In Hatak | OpenAI has just brought us closer to “Her” than ever: her new voice model will accompany us (and maybe make us fall in love)