Get structured JSON data back from GPT-4-Turbo

With the latest gpt-4-turbo model out recently, there is one very helpful feature which came with it: The JSON mode option. Using JSON mode, we are able to predictably get responses back from OpenAI in structured JSON format. 

This can help immensely when building APIs using Large Language Models (LLMs). Even though the model can be instructed to return JSON in it's system prompt, previously, there was no guarantee that the model would return valid JSON. With the JSON mode option now, we can specify the required format and the model will return data according to it. 

To know more about JSON mode, have a look at the official OpenAI docs: https://platform.openai.com/docs/guides/text-generation/json-mode

Now let's look at some code to see how this works in action:

I am using the Azure OpenAI service to host the gpt-4-turbo model and I am also using the v1.0.0.-beta.12 version of the Azure OpenAI .NET SDK found on NuGet here:

https://www.nuget.org/packages/Azure.AI.OpenAI/1.0.0-beta.12

	using Azure;
	using Azure.AI.OpenAI;
	using System.Text.Json;
	namespace OpenAIJsonMode
	{
	internal class Program
	{
	static async Task Main(string[] args)
	{
	Uri azureOpenAIResourceUri = new("https://<your-azure-openai-service>.openai.azure.com/");
	AzureKeyCredential azureOpenAIApiKey = new("<your-azure-openai-key>");
	string deploymentName = "gpt-4-1106-preview"; //Deployment name
	OpenAIClient openAIclient = new(azureOpenAIResourceUri, azureOpenAIApiKey);
	var completionOptions = new ChatCompletionsOptions()
	{
	DeploymentName = deploymentName,
	Temperature = 1,
	MaxTokens = 500,
	ResponseFormat = ChatCompletionsResponseFormat.JsonObject,
	Messages = {
	new ChatRequestSystemMessage("You are a data extraction assistant. " +
	"I will give you some text. You will extract and return the names of major cities in the world from the text. " +
	"Your response should only contain the names in json format. " +
	"The format of the json should be: {\"cities\": [\"<city1>\", \"<city2>\", \"<city3>\"]}}"),
	new ChatRequestUserMessage($"TEXT: In the dynamic tapestry of our planet, bustling metropolises emerge as beacons of innovation, culture, and diversity. From the energetic streets of New York, where towering skyscrapers paint the horizon, to the romantic canals of Venice, where gondolas glide through winding waterways, each city possesses a distinct charm. The vibrant streets of Tokyo beckon with neon lights and technological marvels, while in the heart of the desert, Dubai's futuristic skyline dazzles with its architectural wonders. Along the banks of the River Thames, London's historic landmarks stand as a testament to centuries of rich heritage, while the aromatic streets of Marrakech offer an immersive sensory experience. Whether strolling through the vibrant markets of Istanbul or wandering the enchanting streets of Paris, these cities become the tapestry upon which the stories of countless lives are woven. They are the nodes that connect us, the vibrant hubs where dreams are born, and the melting pots of cultures that shape the world.")
	}
	};
	var completionResponse = await openAIclient.GetChatCompletionsAsync(completionOptions);
	//LLM response is in JSON format {"cities": ["New York", "Venice", "Tokyo", "Dubai", "London", "Marrakech", "Istanbul", "Paris"]}
	var cityResponse = JsonSerializer.Deserialize<CityResponse>(completionResponse.Value.Choices[0].Message.Content);
	Console.WriteLine($"The cities mentioned in the text are: {string.Join(", ", cityResponse.cities)}");
	}
	}
	public class CityResponse
	{
	public List<string> cities { get; set; }
	}
	}

view raw OpenAIJSON.cs hosted with ❤ by GitHub

What is happening in the code is that in the system message, we are instructing the LLM that analyse the text provided by the user and then extract the cities mentioned in this text and return them in the specified JSON format. 

Also important is line 22 where we explicitly specify to use the response format as JSON.   

Next, we provide the actually text to parse in the user message. 

Once we get the data back in expected JSON schema, we are able to convert it to objects which can be used in code.

And as expected we get the following output:

Hope this helps!