Microsoft Corp. announced a major expansion of its artificial intelligence plugins ecosystem today, saying it will adopt the same open plugin standard that OpenAI LP uses for ChatGPT across the entirety of its copilot offerings.

At the same time the company revealed a host of new copilot tools, which are applications that use generative AI to help users with cognitive tasks, such as writing a sales pitch, taking notes or generating images.

The announcements at Microsoft Build 2023 were followed by a wave of AI-focused updates to the Microsoft Azure cloud platform aimed at boosting the productivity of developers and help businesses harness the power of AI for their own ends.

Expanded plugins ecosystem

Microsoft introduced the concept of an AI-powered copilot two years ago when it launched GitHub Copilot, a tool that assists developers in writing code. Since then, it has created additional copilots for services including Bing, Microsoft 365, Dynamics 365, Viva and more. Today, Microsoft expanded this ecosystem to include copilots for Power BI, Power Pages, Microsoft Fabric and Windows, while also unveiling tools for developers to build their own copilots. They include new plugins that enable copilots to become more useful by interacting with third-party software and services.

AI plugins have only recently become a thing, following OpenAI’s introduction of ChatGPT plugins and Microsoft’s plugins for Bing, which debuted earlier this month. Those plugins enable ChatGPT and Bing Chat to help users perform tasks such as finding and booking a restaurant reservation.

“At root, think of plugins as a bridge,” Microsoft shared in a blog post. “This could be a bridge between a large language model that was trained on public data from the internet and all the data that a company may keep privately about its benefits. The plugin is the bridge that gives the copilot access to those files when it answers a question from an employee at the company.”

Now, Microsoft has adopted the same open plugin standard that OpenAI created for ChatGPT for all of its copilot offerings. As a result, any plugin that works with ChatGPT will also be compatible with Microsoft’s copilots, the company explained.

It also means that developers can use a single platform to build their own plugins for services including ChatGPT, Bing Chat, Dynamics 365 Copilot, Windows Copilot and others. As a result, it will be much easier for developers to build experiences enabling users to interact with their apps using natural language, Microsoft said.

Boosting AI productivity and safety

AI is not surprisingly one of the major themes at this year’s Build, and the extended copilot and plugins ecosystems were followed by a slate of Azure-related AI enhancements, designed to help cloud customers boost their productivity and content safety.

For instance, the updates to Azure OpenAI service now available in preview will make it simpler for customers to add their own data to models built using GPT-4, the most advanced large language model created by Open AI. These updates include a new Provisioned Throughput SKU with dedicated/reserved capacity and plugins that make it easier to integrate external data sources.

Azure AI Content Safety, meanwhile, is a new service that helps businesses to create safer online environments and communities, offering AI models designed to detect hate speech and violent, sexual or self-harm content across multiple languages, in text and images. Any offensive content will be flagged with a severity score, allowing human moderators to see which content requires urgent attention, Microsoft explained.

Azure AI Content Safety is being integrated with various products, including Azure OpenAI Service and Azure Machine Learning prompt flow. It will become available on June 1, Microsoft said.

Other updates include Azure Cognitive Search, in preview now, which brings vector search capabilities that allow applications to store, index and search by concept as well as keywords. Among other things, Microsoft said, developers will be able to build apps to generate personalized responses in natural language, deliver product recommendations, detect fraud and identify data patterns.

Azure Cognitive Search for Language, also in preview now, will help developers to interactively customize language skills across their apps, leading to faster time-to-value for organizations looking to use LLMs.

Generally available starting today, Document Translation in Language Studio is a new capability that makes it possible to translate documents in batches, while the Power Automate Translator connector, currently in preview, provides simple workflow automation to translate text and documents from multiple clouds, on-premises or local storage.

Putting generative AI into operation

On the AI front, Microsoft announced a host of updates to its machine learning operations platform Azure Machine Learning that it said are designed to make it easier for teams to put responsible generative AI models into operation.

The updates to Azure Machine Learning include Prompt Flow, launching in preview soon, which will provide a streamlined experience for prompting, evaluating and tuning LLMs. It will enable teams to quickly create prompt workflows and connect these to multiple language models and data sources, with tools to measure their quality.

Support for foundation models is available in preview now and provides native capabilities to fine-tune and deploy foundation models from open-source repositories using Azure Machine Learning components and pipelines. Other capabilities in preview now include support for text and image data in the Responsible AI dashboard, allowing users to evaluate large models built with unstructured data during the training and evaluation stages, and “Model Monitoring,” which provides tools for tracking model performance in production to enable continuous improvement.

Finally, Microsoft announced various other updates to support collaboration, governance and rapid development at scale in Azure Machine Learning, including a managed feature store, Microsoft Purview connector, managed network isolation, support for DataRobot 9.0, Azure Machine Learning registries and Azure Container for PyTorch.

Image: Microsoft