By now, we have seen "Chat with your documents" functionality being introduced
in many Microsoft 365 applications. It is typically built by combining Large
Language Models (LLMs) and vector databases.
To make the documents "chat ready", they have to be converted to embeddings
and stored in vector databases like Azure AI Search. However, indexing the
documents and keeping the index in sync are not trivial tasks. There are many
moving pieces involved. Also, many times there is no need for "similarity
search" or "vector search" where the search is made based on meaning of the
query.
In such cases, a simple "keyword" search can do the trick. The advantage of
using keyword search in Microsoft 365 applications is that the Microsoft
Search indexes are already available as part of the service. APIs like the
Microsoft Graph Search API and the SharePoint Search REST API give us "ready
to consume" endpoints which can be used to query documents across SharePoint
and OneDrive. Keeping these search indexes in sync with the changes in the
documents is also handled by the Microsoft 365 service itself.
So in this post, let's have a look at how we can combine OpenAI's gpt-4o Large Language
Model with Microsoft Graph Search API to query SharePoint and OneDrive
documents in natural language.
On a high level we will be using OpenAI function calling to achieve this. Our
steps are going to be:
1. Define an OpenAI function and make it available to the
LLM.
2. During the course of the chat, if the LLM thinks that to respond
to the user, it needs to call our function, it will respond with the
function name along with the parameters.
3. Call the Microsoft Graph Search API based on the parameters provided by
the LLM.
4. Send the results returned from the Microsoft Graph back to the LLM to
generate a response in natural language.
So let's see how to achieve this. In this code I have used the following nuget
packages:
https://www.nuget.org/packages/Azure.AI.OpenAI/2.1.0
https://www.nuget.org/packages/Microsoft.Graph/5.64.0
The first thing we will look at is our OpenAI function definition:
In this function we are informing the LLM that if needs to search any files as
part of providing the responses, it can call this function. The function name
will be returned in the response and the relevant parameter will be provided as
well. Now let's see how our orchestrator function looks:
There is a lot to unpack here as this function is the one which does the heavy
lifting. This code is responsible for handling the chat with OpenAI, calling
the MS Graph and also responding back to the user based on the response from
the Graph.
Next, let's have a look at the code which calls the Microsoft Graph based on
the parameters provided by the LLM.
Before executing this code, you will need to have created an App registration.
Here is how to do that:
https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app
Since we are calling the Microsoft Graph
/search endpoint with
delegated permissions, the app registration will need a minimum of the
User.Read and
Files.Read.All permissions granted.
https://learn.microsoft.com/en-us/graph/api/search-query?view=graph-rest-1.0&tabs=http
This code get the parameters sent from the LLM and uses the Microsoft Graph .NET
SDK to call the
/search endpoint and fetch the files based on the
searchQuery
properties. Once the files are returned, their summary value is concatenated
into a string and returned to the orchestrator function so that it can be sent
again to the LLM.
Finally, lets have a look at our CallOpenAI function which is responsible for
talking to the Open AI chat api.
This code defines the Open AI function which will be included in our Chat API
calls. Also, the user's search query is sent to the API to determine if the
function needs to be called. This function is also called again after the
response from the Microsoft Graph is fetched. At that time, this function
contains the details fetched from the Graph to generate an output in natural
language. This way, we can use Open AI function calling together with
Microsoft Graph API to search files in SharePoint and OneDrive.
Hope this helps!