Natural Language UI Control: Langchain, Next.js, OpenAI Functions & WebSockets

--- type: transcript date: 2023-08-16 youtube_id: ag9la1e78yo --- # Transcript: Natural Language UI Control: Langchain, Next.js, OpenAI Functions & WebSockets in this video I'm going to be showing you how you can leverage a line chain open AI function calling websockets in your next.js application to build an application that takes in a natural language prompt like this so if I say what is Apple stock doing what this is doing my front end is listening for the websocket events and there's three different events so there's one for the ticker there's one for the stock chart itself and then there's one for the natural language query so obviously you could scale this up from just three to be a whole dashboard if you'd like you could have all sorts of financial information you could imagine in this type of example or just have it render out a view so you could imagine you have an application and you want to incorporate natural language imagine being able to have a user input what they want to see within the application and then it just renders it out so I thought this being an interesting area to explore there's a lot of natural language in natural language out examples out there so I thought I'd dive into an alternative approach with some cool Tech that you can leverage to incorporate into your application so the first thing that I'm going to have you do is we're going to have to set up a handful of API Keys within our project so you can simply do a create next app and then I'll run through all the steps here also shortly after the video is posted I'll make sure to include a GitHub repo if you want to use this as a starting off point so one thing to note I'm going to be showing you basically all the services are free with the exception of the openai API now with that said this example is using the GPD 3.5 model so it is still relatively cheap so the first thing you can do is you can make an account on pusher.com this is going to be what we use as the service to distribute our different websocket events so you can set up an account for free then what you're going to need is you're going to create a channel once you've created a channel you'll have a dashboard view like this and you can simply go into your app keys so once you have your app Keys available you can go within the dot env.local.example in this how it's going to ultimately be outputted we're just going to have our DOT env.local but I'm just showing you this for example sake and what you're going to do is you're going to put all your different values within the variables here so next from there I'm not going to show you how to get these because they're incredibly simple just go over to the openai API page and then you can get an API key and then similarly on Alpha Vantage super simple you don't even need to log in to get an API key if you're using their free tier now one thing with Alpha Vantage they give you five requests per minute with a hundred requests per day so you'll be able to try this out if you want to test it out with this example so the first thing I'm going to do is I'm going to run through the back end and then I'll go through the front end one thing to note I just have two files that we're going to be working out of after our DOT EnV is going to be our route and our page and that's it so it's not a whole lot of code to be able to accomplish this and yeah we'll just get going here so the first thing that you're going to have to do is you're going to have to install a handful of packages so I have the packages that we're going to install in the back end and the front end within the files here so in this example I'm using pnpm so you can pmpm.anv Lang chain Pusher and Zod now with Lang chain once you install it I believe dot envy and Zod are dependencies of the project so you might not need to install them but I just included them here just in case so we're going to include a handful of modules here now as I go through the code I'll dive into the description of what these are doing as we go through them so first we're going to load in our environment variables from our EnV that we just set up then we're going to establish the model that we're going to be using so this defaults to GPD 3.5 but you could specify gpt4 if you'd like then here is where we're going to be reaching in for all our different Pusher variables and then here is going to be our post request route so the first thing we're going to do is we're going to destructure the message that we received from the front end and that's the only input that we're going to be accepting in the back end here then once we have the message what we're going to do is we're going to declare a new Dynamic structure tool so this is a tool within Lang chain and it's set up in a way where you can use it for tools for agents but it's also set up in a way where you can use the what you set up here for openai function calling so there's a handful of things here to dissect now this uh fetch historical data is what we're going to be using to get the chart here so you need a description that the natural language llm maps to what you want to have it do so here I have it trigger stock data based on their ticker and then I have this schema where I specify I want an argument of a ticker which is a string so the Z dot object and Z dot string is Zod so that's what we're going to be using for our schema validation so once we have that we're going to write our traditional function within the method here so here we're going to be specifying our Alpha Vantage API key and this is the route to interact with that time series so we get an array of all those different data points of the date and the value and then we're just going to do a simple fetch request and then we're going to parse that and then from here we're going to just map through it and this is going to be what we send back to our or through our Pusher channel to our front end so our front end is going to be listening for chart and ticker as well as the message here and then I'm also going to be returning a response for our lane chain agent which I'll show you in just a moment here with the value of the last value within that array so the most recent value okay so the next thing we're going to do here is this is going to be where we're going to establish the tools for our agent so as you see here I have one function declared for our tools array here but you could declare you could just copy this and you know paste a number of different functions if you'd like and you can have it all broken up so say if there's different queries and you want them isolated like that you can have a whole bunch of different functions that are passed into the openai API and then here we're just initializing our agent we're passing in our tools we're passing in the model that we set up and then we're specifying that this is an open AI a functions agent so here we're simply running the agent with our message and then finally once this has finished we're going to return the result so you see this message is the last thing that gets rendered it's getting the ticker first once it's parsed so these two will render pretty quickly one after another but it's going to go in that order as you see the different Pusher triggers that have run through here so next we're going to go over to the front end and similar to the back end I have the packages that you need to require right at the top here so pnpmi Pusher chart.js and react chart.js2 so we're going to specify that we're using client on this since it is a client-side component we're going to require some of the standard react things that we need use effect and use State and whatnot we're going to import pusher here we're going to import a handful of things for our chart we're going to register them we're going to import the line chart so one thing to note if you haven't used charges is it's not just line charts there's a whole different or a whole bunch of different charts I should say so you say if you want to have a dashboard View and you have pie charts or line charts and all sorts of things you can leverage this library for a whole bunch of different chart styles so here we're going to be establishing and defining our app component and then from here we're just going to be declaring a number of hooks so we have one for our chart data we have one for our input value of our input here we have our message we have our ticker and then we also have loading States for each of these three so from there we're going to be handling the input change here so as this changes we're going to be setting our value of the hook here we're also going to be handling our key press so we're going to be listening for the enter key so it works on cement and on enter now within our use effect we have a fair bet here so this is going to be where we actually set up our new Pusher instance and we're going to be referencing to our implementation of the channel That We have set up then from here we're going to do a channel dot bind so this is where we're going to be listening for that chart event so if I go back to the route here and I go to the bottom of our function here you see that we have our channel name and then we have the value that it's listening for so that's going to listen for whenever that chart is sent across we're going to log out that data and then we're going to set the state of our chart data and the main thing that comes through the channel is what's within the data here but there's also some things that are just set up for the standard configuration of the chart and then finally once that's set we're going to just change the loading state to false so similarly for the message we're just going to do this essentially the same thing it's a lot simpler because we just have the string as a message and then we're going to change the loading state and then a similar thing for the ticker as well and then finally we'll just do a little bit of cleanup for our websockets at the end here so once we have that we're going to change the states of a handful of things and set the value of our message and all the other values of our different hooks that we've declared and just reset them every time that someone clicks enter essentially is the way that you can think about these so then we're going to have a try catch and this is going to be where we actually send our fetch request to our back end so it's within the route of API slash chat so we're doing a post request we're stringifying the message so we just have the one key we're sending across and then it's just going to Simply let us know if it's sent correctly or not and then finally we're just going to go through our jsx so we're going to have a couple wrappers so we're using Tailwind in this example and it's just like you see visually so the top to the bottom so it's going to be the ticker at the top here it's going to see whether it's loading if it's loading we have that loading State and then if it's loaded we have the ticker itself for the chart same thing loading state if it's rendered it will rendered messages same thing and then finally we just have our input so as you see it's pretty straightforward to set something like this up I thought I'd share an example because I think this is a different area within sort of the AI boom and being able to build apps with llms that is a little bit less focused on but nonetheless very neat so this is something that I just came up with relatively quickly but you could imagine like the different applications for this right so say you have an application and you want to throw in a text box where users maybe set their settings so you can imagine you have different sliders or what have you and based on a natural language prompt just set their settings or you know there's a multitude of examples or things you could try with this so I just thought I'd give you a little bit of a starting off point if you're working or looking to work within this Tech stack but that's pretty much it for this one so if you found this video useful please please like comment share and subscribe I also set up a patreon recently after a couple individuals requested one which was very nice of them so if you're interested in becoming a patreon subscriber I'll also put a link in the description but otherwise until the next one