Google LLC is updating the new “AI Mode” feature in Google Search, introducing multimodal capabilities that allow it to “see” images uploaded by users, so it can better answer their questions.

AI Mode was introduced in a limited preview last month for Google One AI Premium subscribers. It’s an experimental feature in Google Search that users generative artificial intelligence and allows users to ask complex, multipart questions and follow-up queries to dig deeper into a specific topic.

With today’s update, those who have access to AI Mode can now upload images and ask questions about what it sees, Google revealed in a blog post today. In addition, the AI Mode feature is being rolled out to millions of new users who have enrolled in its Labs program to get early access to new applications.

Google said AI Mode’s image analyzing functionality is powered by the multimodal search capabilities in Google Lens, which is a smartphone app that allows users to take photos with their camera and search them within Google Search.

According to Google, AI Mode will be able to understand the entire scene in any uploaded image, including how different objects within it relate to each other. It will also be able to ascertain the materials those objects are made out of, their shapes and colors, and their arrangement, Google said. It will ask multiple questions about the image and the objects within it, allowing it to provide a more detailed response than traditional Google Search does.

As an example, Google said someone could snap a photo of their bookshelf and enter the query, “If I enjoyed these, what are some similar books I might like?” AI Mode will scan the image to identify each book, and then it will recommend a bunch of other books after doing some research on those titles.

In addition, users can ask follow-up questions. So the user might stress that he or she is “looking for a quick read, which one of these recommendations is the shortest?”

Google Search Vice President of Product Robby Stein stressed that “AI Mode builds on years of work in the area of visual search and takes it a step further.”

Given that AI Mode is still an experimental feature, it’s not clear how popular the service has become. Google launched it last month as a response to popular generative AI search applications such as Perplexity and OpenAI’s ChatGPT Search, which provide similar capabilities.

Google has said it will continue to refine the user experience and expand the functionality of AI Mode prior to a more general release.

Images: Google