Microsoft Corp. is updating Microsoft Copilot, its collection of conversational search services, with new features that will enable users to find information and perform data science tasks more easily.

The company announced the update this morning. Some of the new capabilities are available today, while others are set to roll out in the near future. 

Copilot is Microsoft’s recently introduced brand name for two services: the conversational search tool previously known as Bing Chat and the chatbot built into Microsoft 365, the product suite that includes Office. The two services have similar feature sets. They can search the web, draft documents, generate images and write code.

Copilot is based on artificial intelligence software from OpenAI. As part of the update announced today, Microsoft is upgrading the offering to GPT-4 Turbo, the latest version of OpenAI’s flagship large language model.

Compared with its predecessor, GPT-4 Turbo can more accurately understand the instructions that users include in their prompts. This makes the model better at tasks such as generating code that require a high degree of accuracy. Moreover, GPT-4 Turbo enables users to include significantly more information in each prompt than OpenAI’s previous flagship LLM.

In Bing, GPT-4 Turbo will not only process standard user requests but also power a new feature called Deep Search. The latter feature can analyze a user’s search query and rewrite it to add more detail. In some cases, Deep Search generates up to several sentences’ worth of additional information. 

According to Microsoft, the extra detail the feature adds to user queries allows Bing to surface more relevant results. To increase the usefulness of the retrieved information, the search giant also draws on more data sources. Microsoft says Bing, which considers “millions” of web pages when processing standard search queries, analyzes 10 times more content if Deep Search is activated.

“Deep Search harnesses the power of GPT-4 to deliver optimized search results for complex topics,” Yusuf Medhi, Microsoft’s consumer chief marketing officer, detailed in a blog post. “Activating Deep Search expands search queries into more comprehensive descriptions to deliver more relevant results.”

Deep Search is rolling out alongside another new tool called Code Interpreter. According to Microsoft, the tool can generate code and perform data science tasks such as averaging the numbers in a spreadsheet column. OpenAI offers an identically named feature as part of its ChatGPT chatbot. 

Code Interpreter performs data science tasks by writing and then running Python code in Azure Container Apps, an application hosting service available through Microsoft’s public cloud. Python is widely used for data science projects because it contains many libraries, or prepackaged code components, that can perform statistical calculations. Those libraries remove the need for AI models to generate all the code needed for a project from scratch. 

Microsoft says Code Interpreter can perform data science tasks both on information retrieved from the web and files provided by users. A consumer could, for example, upload a spreadsheet containing product prices and ask the tool to add them up. Users can optionally have Code Interpreter visualize the uploaded data in graphs.

Multimodal processing is another major focus of today’s update. Copilot now enables users to include photos in search queries and will provide access to a new version of DALL-E 3, the OpenAI model that powers Bing’s image generation feature. The upgraded model can generate higher-quality images that more closely match the description provided by the user.

Microsoft’s push to enhance Copilot’s multimodal capabilities extends to other use cases as well. In Edge, the embedded chatbot can now answer questions about a video or generate a summary. When viewing a webpage that contains text, users can select a paragraph and have Copilot automatically rewrite it.

Image: Microsoft