Building an Agent for Microsoft 365 Copilot: Adding an API plugin

In the previous post, we saw how to get started building agents for Microsoft 365 Copilot. We saw the different agent manifest files and how to configure them. There are some great out of the box capabilities available for agents like using Web search, Code Interpreter and Creating Images.

However, a very common scenario is to bring in data from external systems into Copilot and pass it to the LLM. So in this post, we will have a look at how to connect Copilot with external systems using API Plugins.

In simple terms, API Plugins are AI "wrappers" on existing APIs. We inform the LLM about an API and give it instructions on when to call these APIs and which parameters to pass to them. 

To connect your API to the Copilot agent, we have to create an "Action" and include it in the declarativeAgent.json file we saw in the previous post:

The ai-plugin.json file contains information on how the LLM will understand our API. We will come to this file later.

Before that, let's understand how our API looks. We have a simple API that can retrieve some sample data about repairs:

Next, we will need the OpenAI specification for this API

The OpenAPI specification is important as it describes in detail the exact signatures of our APIs to the LLM.

Finally, we will come to the most important file which is the ai-plugin.json file. It does several things. It informs Copilot which API methods are available, which parameters they expect and when to call them. This is done in simple human readable language so that the LLM can best understand it.

Additionally, it also handles formatting of the data before it's shown to the user in Copilot. Whether we want to use Adaptive Cards to show the data or if we want to run any other processing like removing direct links from the responses.

If you are doing plugin development, chances are that you will be spending most of your time on this file.

Once everything is in place, we will run our plugin with simple instructions and this is how it will fetch the data from external database

Hope this helps!