Google LLC today announced an upgrade to its conversational artificial intelligence search tool, AI Mode in Search, expanding its capabilities.

The company introduced AI Mode in March, which lets users transform the familiar Google Search experience from simple lists of links into dynamic, personalized explanations and cohesive paragraphs, while also enabling them to ask direct questions about web content.

With the new update, Google is adding four big features across Search Live (video with Google Lens access), Canvas, image and PDF uploads and more.

Users can already ask AI Mode on mobile devices in the Google app about images; this week, the same capability is launching for desktop browsers. The company said that in the coming weeks, the same capability will roll out to support PDF uploads.

This allows users to ask complex questions about lengthy PDF documents, including legal files, extensive financial data or slides from academic lectures. Users can then use the AI chatbot to ask follow-up questions to deepen their understanding of the content beyond the base material. It will cross-reference the vast information on the web to provide helpful AI responses, along with citations to allow users to dig deeper.

Google said in the months ahead, the company intends to add further support in AI Mode for other file types beyond PDFs and images, including files in Google Drive.

Project building using Canvas

Canvas provides a way for users to build projects and plans over multiple sessions in a dynamic side panel that updates as they progress.

For example, if a user is searching for information on the web to create a study guide or is building a project, AI mode will ask if they want to make a Canvas. If the user clicks the button, the chatbot will immediately start building a Canvas side panel with information from searches and will use the conversation to refine the information and formatting.

“No matter what you’re working on, from test prep to travel planning, you can always come back to your Canvas project and pick up where you left off,” said Robby Stein, vice president of product at Google Search.

Canvas will become available for users in the United States over the coming weeks who are enrolled in AI Mode Labs experiments.

Real-time video helps with Search Live

This week, Google is launching Search Live with video input, which will bring advanced capabilities from the company’s Project Astra directly into AI Mode.

Project Astra is the company’s work on building a real-time multimodal assistant that can both see and speak to the user. Users can launch it from within Google Lens with the new “Live” tab alongside “Search” and “Translate.”

Once running, AI Mode will be able to view what the mobile camera can see and respond verbally to questions about what’s happening.

“When you go Live with Search, it’s like having an expert on speed dial who can see what you see and talk through tricky concepts in real-time, all with easy access to helpful links on the web,” said Stein.

The new feature will become available to iOS and Android users in the U.S. for users enrolled in Mobile Labs experiments.

Google Lens coming to desktop Chrome

Google Lens enables users to search the internet using images or the camera on their mobile devices. Now, with its integration into Google Chrome on desktop computers, users can inquire and learn more about what they see on their screens — whether it’s a website, a PDF or any other content they’re viewing.

This feature already exists for Windows 11 users with Copilot Vision, Microsoft Corp.’s AI chatbot that lives in the operating system, which can see the desktop and answer questions about it when summoned.

Beginning soon, when users click on Chrome’s address bar, they’ll see a new option to “Ask Google about this page” in the dropdown suggestions, which will allow them to access Lens in Chrome. This will pop up a sidebar with an AI Overview of selected images or the desktop.

This week, that capability is being expanded with AI Mode, allowing users to dive deeper, follow up with questions and receive expert information about what they’re seeing.

Image: Google