Answer Engine: Groq Function Calling - Next.js, AI SDK, Mixtral, Langchain, OpenAI, Brave & Serper

--- type: transcript date: 2024-04-12 youtube_id: kV2U7ttqE-g --- # Transcript: Answer Engine: Groq Function Calling - Next.js, AI SDK, Mixtral, Langchain, OpenAI, Brave & Serper in this video I'm going to be showing you how you can set up your own conditionally rendered UI based off natural language by leveraging the new Croc function calling capabilities as well as the versel AISD kit I'm going to be going over three different examples I'm going to be showing you how to set up a map with different locations by leveraging the server API going to be showing you how you can set up a stock chart that's interactive from Trading view which is completely free going to be also showing you how to set up a simple shopping component so if you didn't see my previous video where I built out this answer engine this is really building off of that I just genely want to thank everyone for their support over these past few weeks this repository really took off in a way that I was not expecting I've had different creators create content about the video which hard to believe I've had all of a sudden a different company's reach out to me I'm just glad that I'm able to provide something useful where it's something that either you can build with or learn from do you just like using the tool my plan right now for the project I'm going to be building out some features that I haven't yet seen in any sort of other chat interface yet and show as examples I'm just excited to see where this project will go now the first thing that we're going to do within our project we now have this configuration file which I have commented glore here the first thing that we're going to do to set up this new function calling capability is we're going to set up this key value pair within the configuration object I have this key of use function calling and you can simply turn it on or off and then adjacent to the action and config we're going to make a file called function calling right now I'm going to put all of the functions as well as the logic that orchestrates the function calling capabilities from the llm all within one file just to show you for demonstration sake we're going to be using the open AIS SDK I still have it set up to groc so we are going to be using that new groc function calling capability if you do want to use something like GPD 3.5 you can use that as well what we're going to be doing is we're going to be setting three simple functions and the thing with these different functions is if you want to swap these in for some of the components that already exist like the images component or the videos component so let's say you don't necessarily want to always render images or videos you could just throw them in here and have that logic be conditional the first function that we're going to be setting up is the search Place function the end state of this is we are going to have an interactive map that we can use the map itself we're going to be using leaflet which is a completely free open source Library we don't need to pay for anything like a Google Maps API key or anything like that and then we're going to be leveraging the already existing server API that exists within our project if you already have the project and you just want to bolt this on into the project that you already have you can leverage those API keys that already exist within our first function we're going to specify that we want a query as well as a location and this is going to be how we render things like an Italian restaurant in New York or a bagel shop in San Francisco or whatever it is right so we need those two different queries within this particular function within the function we're just going to Ping the server endpoint and we're just going to send in our API key and then we're also going to be sending in that query and location that we had just talked about once we have that we're going to normalize the data a little bit and then we're going to add this key of places now the reason we're going to be doing this is going to be how we later route those different functions on where they need to go within the UI once we have that we're just going to set up a catch if there are any errors when we invoke this function next very similarly we're going to call that serper API to go shopping if it detects that the user quote unquote wants to go shopping we're going to specify that we need a string for this function and then it's going to detect from the query on whether there's something like Patagonia vest or Ugg boots or whatever you put in here almost identical logic we're going to be setting up the request the exact same way and then in this one we're going to be adding that key of shopping and then we're just going to be returning all of those results similarly we're also going to log out if there are any errors when interacting with the API and then in our last function we're going to have a get tickers function which is going to specify a particular ticker so if we say I want to know more about Apple stock or Microsoft stock or whatever it does all we're going to do within this is we're going to again add that key value pair of type we're going to specify that it's ticker and then we're going to stringify the results next this is going to be where it really orchestrates the function calling capabilities when it's interacting with the llm first we're just going to declare our system message here I'm just going to specify to make it even further and hopefully more compelling to the llm I'm going to say you are a function calling agent you will be given a query and a list of functions your task is to call the appropriate function based on the query and return the result in Json format only call a function if you are highly confident it will be used this is the the point that I get to in programming where you just honestly have to laugh that this is what you all of a sudden have to do with interacting with these llms maybe if it's uncertain that it needs to use location data hopefully this system message is weighted a little higher where it will recognize it and it won't call it unless it's really certain from there we're going to be passing in the query that's going to be what in the input box on the front end and then here is going to be where we specify the different tools that we're going to be sending in because we have three different functions we're going to have three different mappings within the tools here so this is the particular schema that you have to follow to set up function calling you're going to specify that it's function and then within the function you're going to be describing what the function does the function name the parameters that it has what arguments that it has what type of arguments that they are whether it's a string or a number or what have you optional or required all of that stuff is within these different tools here you can look at these and play around with these and potentially use these and tweak them they're all found within here and they all map to each of these individual functions here once we have that's going to be where we actually specify our chat completion here we're going to be passing in the model that we have specified within our configuration file then we're going to be passing in the messages here that we had just declared we're going to be specifying the tools that we just went through and then we're going to be specifying that the tool choice is Auto and then here you can optionally choose the max token length as well now one things that I found is the functions that I'm using none of them really require long arguments right it's going to be like three or for different letters if we go to the search places all that we're going to have is the city and E query we have two arguments but again we're just going to have the query and then we're going to have the location and then for the go shopping function we're just going to have the query so if you say Patagonia vest it's going to say Patagonia vest so the payload that you're actually getting back is really not that high so this should be more than enough tokens that you need for your function invocations unless you're doing something really adventurous where you have a ton of different things here the next what we're going to be setting up is once we have that response message we're going to have it within this response message variable now within the tool calls essentially what we're going to be doing is we're going to be looping through the different tool calls that were detected if there are multiple and then in here we're just going to go through and see which ones were detected in this application like I mentioned I'm only really expecting to call one function but you could set it up in a way where you have multiple functions that are being called so here if it detects get tickers we just just going to go ahead call that function with the payload that we got back from the llm so this would be like something like e l for Apple sticker If it search places we're going to have the query and the location all of that stuff that we had mentioned previously and that's pretty much it so this sort of orchestrates that logic on how I set this up once the video goes live I'm going to probably make a few tweaks over time within this section here but that's pretty much it for the function called files I've made a couple tweaks to the action. TSX and basically all that I've done is within our promise where we're concurrently getting the images sources videos now we're also going to be doing the function calling but you could also take this out and move it below and do something like I have within the streamable update here so we could say something like if config do function calling and then within that we could just specify we want the function calling response to be the user message and then we could have the streamable response there if we'd like and then that way it's just adjacent but you are going to be waiting longer if you have it set up like this but that way the whole configuration could just be wrapped you can turn it on so you have that flexibility if you'd like but I'm just going to move it back to the way that I have it and then just save it out here so you can play around with these a little bit depending on what you'd like to see or how you'd like to have it configured then within the page I'm going to go through this relatively quickly but the best way to go through and and see what I did to set this up is if we look at some of the new components the shopping component for instance so you see all of the different types that we have for the shopping component now you can just go through and you can search shopping on the new message we have these new keys the main portion on how it determines where in the UI it's going to go is we have the similar structure that we had in the previous video on how we determine where the different things go here we have this set up so if it detects that there's conditional function call UI we're going to go ahead and parse that and we're going to look for the different keys so whether it's places shopping or ticker it's going to go ahead put it within state within our application for places within that grouping of messages we're just going to push it on the places key and then with our messages map going to pass in the props of places into our map component or for instance if it's our shopping component we're passing the props into our shopping component and the way that I have this set up is you can really move these around so say if you want the shopping component on the secondary content area or the sidebar you can have it there or if you want it right at the top results you can go ahead and save it up there you can really play around with these things there's like Lego blocks you can move them around however you want they're set up in a way where if they don't come through they're just not going to render so that's pretty much the main changes that I made to the page. TSX and then just for the actual components themselves you see within my terminal here you see all of the new files that I have here within the project so we have a handful of new components here so I have them in the answer directory just like I have the other ones so within here we have five new components we have a financial chart from Trading view we have the initial query which is the initial queries that we have on the screen here you saw in the original example we have our map we have our map details where it shows the details of the location on the right hand side there and then we have our shopping component take a look at how they're set up some of them are more simple than others and some of them are just interesting to use but hopefully it just serves as an example of different things that you can add within the project so if you're interested and you have some ideas and you want to take a stab at throwing in some interesting components within the project make a poll request submit an issue when I have time I'll try and add some of the features that people have requested but otherwise that's pretty much it if you found this video useful please comment share and subscribe and otherwise until the next one