Ollama Structured Outputs in 5 Minutes

--- type: transcript date: 2024-12-08 youtube_id: KXQU3mJTvuw --- # Transcript: Ollama Structured Outputs in 5 Minutes we now have structured outputs within ol I was really excited to see this I'm going to read through the announcement here I'll show you some of the examples AMA now supports structured outputs making it possible to constrain a model's output to a specific predefined Json schema this is available both within the olama python as well as the JavaScript libraries in terms of some of the examples for structured outputs here parsing data from documents extracting data from images structuring all language model responses more reliability and consistency than Json mode to get started you can download ama if you haven't already if this is the first AMA video that you're watching you can get it completely for free just go to ama.com follow the steps to install it it is very straightforward in terms of the dependency and the library itself you can pip install Ama or you can mpm install AMA just like it shows here depending on how you're going to be using this within here they have a curl example if you just want to go and try this out right away now you do just have to make sure that you do have the latest version of olama installed there is a chance that you might just have to restart AMA to make sure that you have the most recent version basically what structured outputs allow you to do is you can specify the format and schema of the object and what you'll do is you'll map with natural language the different fields that you have in this case we're going to have the name Capital as well as the language we have two strings for the name and capital and then for the languages if it's a country with multiple languages we're going to have an array in this particular format so we're going to have an array of items and each of those items are going to be strings you can also specify which of the structured fields are required in this example you can specify that all three are required if you ask for tell me about Canada it would return with this schema here so the capital being Ottawa the languages being English and French and then the name of the country are all there if you're leveraging python it's leveraging the pedantic library to get these structured outputs the way that it's set up and recommended from the JavaScript example is it's using Zod as well as Zod to Json schema for the libraries and to determine what you want as a structured output it looks just like this you have this really nice implementation where you can specify the name is going to be a string Capital string languages is going to be an array of strings right and then to call it you can just invoke it the open AI SDK example of what data extraction looks like say if you have a message I have two pets a cat named Luna and then I also have a 2-year-old black cat named Loki and if you go down here we go to pets we have an array of pets their names we have the animal type we have the the age the color as well as their favorite toy this could be used to start to collect information whether it's within a chat based application or whether you're actually using this within some pipeline to extract information from websites documents or what have you now the other cool thing is AMA also supports Vision on some of the models on llama 3.2 vision for instance say you want a summary of the photo but in addition to the summary you also want things like the names of the particular objects in the photo within that photo we had a tree we had a beach in this example it's also self-determining the confidence of that object that they named also some of the attributes of that particular object and in addition to that you can really push this pretty far and get a lot of other details the scene is a beach the colors are blue green and white the time of day is the afternoon setting outdoor so on and so forth another great thing with this is it's open AI compatible if you're using the openai SDK whether it's the typescript library or the python implementation you should be able to leverage that SDK and just be able to swap out the pieces that you need to point it towards olama to do that within the client you just have to make sure that you point the base URL to where you have olama hosted it defaults to this Local Host 11434 port and you do have to specify an API key I don't actually think it needs an API key it just needs something to fulfill that there is a valid string and then for the model of course you can just swap in the model that you're using for the particular use case there are a couple tips that they mentioned within here use the pedantic or Zod Library like they showed within the examples that I just went through they also recommend to put within the prompt to return as Json so it better understands your request now the other thing to consider is to set the temperature to zero this would give a more deterministic output