Langchain, Next.js, Brave API, OpenAI GPT 3.5/4 and Vercel Postgres

--- type: transcript date: 2023-06-07 youtube_id: CXVGhaBUFgs --- # Transcript: Langchain, Next.js, Brave API, OpenAI GPT 3.5/4 and Vercel Postgres in this video I'm going to show you how to set up and deploy a custom full stack autonomous agent using Lang chain open AI with up-to-date data from the brave API so the tech stack that we're going to be using for this is going to be node.js and next.js to build out our front end and server and then we're also going to be leveraging the new versel postgres database to be able to store the the information that we get from our llm so we're going to be using the openai GPT 3.5 API and then finally we're going to be incorporating the cron jobs from versel so essentially once this is deployed it will just run on its own at a specific interval so as you see here what I have set up basically what it's doing is it's reaching out to Brave it's getting the breaking news for the any the given time that it's queried it's normalizing that data so it gets it within a nice format it's all strings we're not passing in extra tokens like you know a Json format to the llm so we normalize it in a nice format for the llm we include the links to be able to have these annotations so we can link out to the story so if you want to see okay of this story that it generated what's the actual news article if you're interested and then from there we would also have the option to pass in a character so say if you want to have the news within the story of Romeo and Juliet or characters of The Lord of the Rings I'll show you how to pass in a query parameter where you can have that as well as an option and then finally once we have that result back from the llm we're going to store that within the versaille postgres database so we're going to go through a handful of things there's going to be a few assumptions here so hopefully you've worked in an xjs if you haven't that's all right but hopefully you've worked at least in react so the front end as you can see it's not the biggest part of this application it really is more back-end uh heavy so if you're comfortable with node.js this you don't worry about not knowing next or react or react or anything like that you should be able to follow along just fine and I'll try and cover everything one by one that you need to get this going so I'll hop on over to my vs code here and so the first thing that I'm going to have you do and I'm not going to run this run every command step by step to show you I'm going to go through what I've sort of generate rated here just so I don't make typos and to sort of for brevity's sake for for YouTube so the first thing I'm going to have you do is create a new next project so you can MPX create next app you'll have this scaffolding and then the one thing I ask of you is just to match the structure that I have here so within here so you'll see within the source and app I'm not using anything here instead I'm overriding everything that's in there and just remove that so you don't have any conflicts or errors that come up we're going to have Pages API generate this is for our API route this is going to be what we hit with our Cron job then we have our index for the um the actual page that you just saw there and then the utilities function is going to be the heavy part of this course and this is going to be where you'll hopefully if you're interested be able to go in there grab the bits and pieces that you need obviously this is a bit of an arbitrary example but hopefully there are some pieces in here that get the gears spinning on how you could potentially develop something similar or with uh the the type of tech that I'm using here so one thing to note with this is if you are concerned about cost so just to give a initial out of the gate broad overview so for the cost from Brave I believe it's nine dollars per I believe it's 5 000 API calls somewhere within that ROM there's different tiers you can look at depending on your needs so very cheap you know like pennies for for queries or fractions of fractions of a penny per query and then similar for GPD 3.5 so with the amount of tokens that we're going to be passing in and retrieved it's also going to be pennies fractions of pennies to generate these at an interval then finally for versel itself so I'm right now I'm on a trial for the proversal you don't need prover cell necessarily but the caveat with that is if you're going to be integrating cron jobs into your application you are severely limited on their free tier so I think you get one or two cron jobs that execute every day so uh you know you won't be able to execute something every 10 minutes or anything like that but you will be able to set up a postgres database for free you'll be able to deploy it for free on versal but you will have to make some tweaks if you're really just you know tinkering around and want to have a a hobby project so you can keep this really really cheap you know like a couple bucks a month if you really uh you know limit the the um the API calls but otherwise um yeah just something to be aware of so with that I'm going to dive into the code here so once you've matched this structure so the main pieces that we're going to be working in are the pages so generate index within the utilities you're going to need a DOT EnV and I'll show you all the variables that you'll need then from there we're going to have our versell.json so once you've set up your next app if you haven't used versel go ahead and install Versa once you have it you can just within the command line type for cell and then go through the options to set up your project the one PC will have to go on their GUI for they're likely are ways you could potentially do it from the the CLI or maybe even from node is to make and instantiate that postgres database so we're going to be doing that part I'll show you where to go ahead and do that from the CLI you'll need a handful of things in there for your environment variables so I'm just going to go ahead and get started here so the first thing I'm going to have you do is open up your package Json so once you have this open you're going to see there's a handful of things that next comes with like next react react Dom I'm going to have you install all of these things above so I'll just quickly run through these now I'll touch on them again as we get into them one by one but we have a motion so we're going to be just writing some really simple CSS I'm not gonna it's gonna be a framework CSS framework agnostic so I'm not going to be using Tailwind or bootstrap or anything you can go ahead and use whatever you're comfortable with and then just remove this styling after the fact if you want it's going to be a very basic though then we're going to have the versel postgres so this just came out a couple months ago it's really easy to interact with once you have it set up but the main part is you just have to follow those steps closely to make sure it is set up properly and once it's set up it's a breeze to work with so from there we're going to need a body parser and actually looking at this we don't necessarily I did some refactoring from right before uh recording this and we're actually going to be making a get request instead of a post request for our Cron job so you don't have to include body parts or you could if you end up making a post request on your server then we're going to be using Dom purify so the reason why we're using Dom purify is to purify the HTML that we're going to be setting within our next app so we're going to have to be using a method called dangerously set HTML and sort of long story short this will help prevent with some scripting attacks with what we have so that's what we're going to be using Dom purify for for DOT EnV environment variables JS Dom similar to Dom purify where we did a little bit on the front end just to work through having the llm return us HTML so we're doing a little bit of uh unconventional piece where we're adding HTML into our react so we are touching the Dom a little bit um and there's a reason for that that I'll get into as we go through here so from that we're going to install Lang chain and that'll be it so just npmi you can do that all on one line and if you forget something you should have a you know an error sort of screaming at you in the terminal once you go ahead and run this so I'll go through the environment variables a little bit and just get you started on what you need so for the brave API you will have to go in and attach a credit card there is a free tier that you do that you can use so I'm on the paid version you might be very limited on the free tier in terms of what you're able to do and the number of queries and whatnot um then from there so go ahead I'm not going to show you like where to get all these API keys to just do a Google search Brave API you know and then just go through the process put in your payment information and whatnot similar for open AI just Google open AI API then once you set up an account you can go to the top right hand corner click your initials and then generate an API key and then finally the versel cron key so I'm going to be exposing this one I'm going to be changing this after this video goes live uh so if you see it you know don't feel like you can go and take that and do something malicious with it because I'm going to be changing it but that's going to be a a key that you set up so you can come up with a value however you want to generate it and put it here then all this postgres stuff so it looks like a lot but we're just going to be copying that from versel so I'll head over to versel and just sort of show you what it looks like on the GUI if you're not familiar so the environment variables we will have to put both in our DOT EnV for when we're developing locally and then when we deploy you have to remember to go into the settings tab of the project environment variables and one for one include all those values here so they're exactly the same and that yeah just just one thing to note uh don't forget to include the so this is the part where I'm going to show you the API key um so this is the Cron job so the logic that we're going to be pinging and setting up in a moment this is the path that it's going to essentially just hit every 10 minutes it's just going to do a get request go to the generate route it's going to pass in the API key and then we're going to query in breaking news and like I mentioned on the outset and the video you can also passion pass in rather an optional parameter of characters so say if you wanted characters like Romeo and Juliet or uh you know Lord of the Rings or whatever Harry Potter whatever you want to put in there you can pass it in and it will give you an optional prompt so I'll show you how to do that as well if you'd like that and then we'll also be setting up the interval so well I'm actually on the Cron job I'm gonna hop over here we're gonna go to versel Json and this is going to be what we have in our versel Json so pretty self-explanatory we're gonna have only one Cron job in this case uh you can have more if you want if you want to do you know you can imagine an application where you know you you have several of these um uh on the go so you can have these at different intervals and whatnot so in our case this is going to be every 10 minutes you can just look up a cron generator if you want to say set it every day or every hour or whatever you want I might set for 10 minutes and then simply we're going to put in the path here and then the API key so this shared key just to go over this just so it's Crystal Clear is going to be our versal kronkey okay so once we have that if you want to set up storage on Versailles just head over to the storage tab you'll see here I have this demo DB instance that I have set up and then within here once you go through and set up the storage you can go over and just finding where it is here if you click dot env.local you can copy this snippet and then just paste it right at the top of your dot EnV so once you have that you've got a brave API key you got an open AI API key and then you generate your own versaillon you can sort of close and save out this folder and forget about it okay so what we're going to do first I think it makes sense likely to go through our utils so this is going to be the part where I think a lot of people will hopefully find Value you can take these functions and change them and hopefully use them maybe as a boilerplate for what you want to get going we're going to go through a lot of the different dependencies uh in this one and sort of the core logic that we're gonna going to need and these uh utility functions you can sort of they're Port very portable so you can bring them over to your application and so long as you you know have the right environment variables and whatnot they should they should work environment variables and dependencies I should say so okay so the first thing we're going to do in the utils is we're going to install all the dependencies so I went over these just a moment ago but just to touch on them again so we're going to use the postgres SQL database we're going to be using dot EnV to reach into our environment variables and then we're going to be using a few things from openai so since we're using the open AI GPT 3.5 turbo model it does have a few extra parameters and then they're old text DaVinci 3 Model so there's other things that you can play around with and I just thought it would be nice to set it up this way rather than just having like one big prompt so it makes it a little bit cleaner and more easier to tweak if you want so once we have all that set up we're going to initialize our DOT EnV config then in here I'm going to be using the brave API endpoint so one thing to note I did see as I was building out this project so Lang chain does have a wrapper around Brave so you could sort of take this function and if you wanted to tweak it and use the Lang chain version you could do that this is sort of just like the raw API version though that I'm going to be showing you So within the brave query uh it's going to be triggered with the query that you saw within that Cron job there and then from there we're just going to be console logging out so I'm going to be console logging out for a couple reasons so one to show you what's happening and hopefully have you you know see as it's running through whether you're in the development um or you're developing locally you can sort of see the steps that if something stops you sort of know which area to look in but then also if there is an issue once it's deployed versel will log out what your console logging so say if on the Cron job it logs out you know uh these two console logs but not the subsequent ones and it times out or what have you um you'll you'll get a context on what you might need to tweak or adjust so once we have that we're just going to encode the query here so it's in an acceptable format and then we're going to pass in the query of freshness PD so this will give us news from the past I shouldn't say news it's not limited to just news but information posts from the past 24 hours and there's a few different options you can see in the brave API docs so we're going to set a couple headers this is going to be how we pass in our brave API key that we included then from there we're going to establish our query here and actually fetch the data so we're going to set up a error handling if there are any issues we're going to return it and make it in a format that we can use within our application and then from there we're going to extract the descriptions from Brave so I'm going to sort of go through this extract descriptions from Brave option really quickly it's it's a whole lot of code but all you sort of the tldr that you need to know about it is it's essentially taking that Json and making it with strings and links so it's think of it as the description from the website and then the link and it's just like going through and making it and removing all the extra tokens for the llm because if you if we just passed in raw Json we'd have a lot of curly brackets and commas and stuff so this it sort of limits our token use is why we're using this extract descriptions okay so uh like I mentioned I'm not gonna go through all of this but you can sort of see the schema that we're going to be working with it's quite a bit of information that you get just from one query so another thing to note we're just gonna be pinging Brave once and this is the neat thing with this we can get a ton of up-to-date information with the brave API and this is just really normalizing that whole response here okay so then from there we're just going to return that out and that is going to be what we essentially pass into our subsequent function here so in here we're going to say I call the function generate fictional story with openai so not entirely fictional necessarily you know but say if you wanted the characters option you know it's potentially more fictional so here this is where we have that optional characters prompt like I mentioned so if you include that character's query in your Cron job so over here in versaille and you added a Ampersand characters and you typed in your characters there that will be what triggers this this to be passed into the lll so once we have that so and just to sort of run through it well this it's all it's doing is adding uh include characters from and you can imagine Romeo and Juliet in the prompt so we're going to be using uh GPT 3.5 turbo I'm going to be setting my temperature to uh 0.7 so from what I've read 0.7 to 0.9 sort of a sweet spot for Creative tasks but you can dial this down if you want so by dialing it up from my understanding it can take longer with the API dialing it down it takes less time so if you run into errors with timing which we'll get into in a moment you can ratchet this down similar to tokens you can also ratchet that down if you run into issues so in this here is where we structure our prompt and so what I'm going to be saying is I want to create a story and a Shakespearean oh shakespean Shakespeare shakes here Burien words are hard uh in a Shakespearean style about the latest news so exclude any mention of news providers and only include information about what seems like uh current events and ins instead write only about news itself that might seem worthy of debate so hopefully it latches on to stories that might seem interesting and then from there you can also pass in your character prompt here so then I put in some notes to sort of have extra emphasis when it's passed and I said Pay careful attention to what's below I want the links to correlate after each news item so that's going to be how we get that annotation for each link so and then I also specified I want no more than five sentences it's not always going to be exact but that will sort of help so we don't have cut off responses and have it be you know a bit more straight to the point and brevity and part of the reason for this is so on versal you do have a Timeout on their serverless functions so where we're going to be querying this endpoint and you have 60 seconds so sometimes you will see that open AI for whatever reason maybe like the gpus are running hot or you know there's just not enough or whatever the reason is it can take a longer time to return so we're going to be passing in a decent amount of data but we're sort of wanting only a little bit of data back so I found this as a little bit of a sweet spot queries might take about 40 seconds or so from what I've noticed occasionally they do time out but because we have that Cron job likely you will have enough but if you if it's really important to you that it hits every time the serverless functions might not be the route to go until you know there's more gpus online and the apis like you know the speed is reliable from it okay so from here then we have our human chat message then I'm just gonna say okay here's the most recent news data pick no more than five of the most relevant stories a user would find interesting then that's where we pass in our brave normalized data there okay so from there we're going to go ahead and wait for the response and then once we've got the response back we're going to return from the response object the raw text from the llm and in this case it's not really raw text because we're also adding it adding some HTML here which we'll get into on the front end so then from here we're going to create our function to create a table so you could just in one line easily query your SQL database like this but this I found just useful just to be able to not have to specify that query and especially if you built out this app and you want to have just sort of standardized ways to interact with your postgres you can have these sort of helper functions but as you can see here it's super simple to use the the postgres from Versa once it's set up so we're just going to set up a handful of these so clear database as you're building out so say you're playing around the prompts you just want to clear the whole thing you can uncomment out the clear database and comment it back this just shows you the SQL to actually wipe that from the database then we have the um so this is also a helper so if you want to initially create the database so the first time you can create the table so the first time just run this then you will have that table created in in Versa and then this is going to be the one that we're querying quite a bit so we're going to be passing in our HTML and then we're going to have our date as well so this is going to be what we call to render that whole view from our our client and the one thing to note with our client we're going to set it up in a way so it's only fetching every 10 minutes which we'll get on into into the next part so it's not going to be every time someone hits the site there's an SQL query I'm going to show you how to sort of get around that so you're not running up your database then from here this is also another method that you could use in a few circumstances so say if you wanted to thread a conversation in the llm and you wanted uh you know the latest response included in the prompt you could include this within your prompt and then we're just going to be exporting these to be able to use these in our application so next what we're going to do is we're going to go through the generate API so this is going to be the Cron job that we interact with and this is going to be what runs every 10 minutes or whatever interval you have had set so from there we're going to get the shared key from the versaille cron so this is going to be to first check to make sure there is an API key so people aren't just querying this endpoint then once we have that we're just going to only allow a get method so we're just going to send back an error code if you send a post request or anything like that just is sort of a nice to have so from there we're going to extract the required parameters from the query string so we're going to extract characters query and API then from there we're going to verify the API key so if there isn't one or it's if it's invalid it will just throw that error so you know where to look for so from there we're going to perform our brave query first so assuming all else is good and we'll get to the API it will perform that brave query once it has that query we're going to go ahead and it's going to normalize within that brave query and then it's going to return in a format that we can pass to our generate fictional stories with open AI then we're going to pass in the characters if they're there then from there we're going to save the response to our SQL then finally we're actually going to also return this to the front end so if you're in the browser and you just want to play around with that endpoint and make sure it's working you know say on production or on dev you can see the response so say if there's an issue with brave or the llm you'll see you know what information it was fed and whatnot then finally we'll just log out if there's any errors so from there in our next application what we're going to be importing is a DOT EnV we're going to be using Dom purify JS Dom emotion and we're going to be requiring the get all HTML strings like I just touched on so I went through these a few times but there's just some comments here if you'd like just to sort of be clear you can read through them so this is all the styling that we're going to use so it's a CSS njs framework you know there's mixed feelings on this thing but if you don't like this just rip this out use Tailwind use whatever you want this is just for example sake so from there we're just going to set up our Dom purify so this is going to be because we have the a tags within the application and then from there we're going to get a static prop so the reason why we're using get static props is this is going to be what revalidates and creates that static version of the site every 10 minutes so you can increase this or decrease this as you need um and that's going to be how we limit our SQL calls so within here we're going to just do some normalizing we're going to get the date within a readable format and then once we have that all set up we'll pass in our props from our static props here into our front end and this is essentially all our front end so these wrappers are our styling we have our title we're just looping through them we have the chapter number I just have an entry number but say in a story context you know you could change a chapter if you wanted and then we're just going to set the HTML so once you have it set up you can deploy it to production you can versl space dash dash prod that will deploy it to the server and then from there you should have a working application that looks just like this so if you run into any issues let me know in the comments I'll put this up on GitHub you can also open issues there ask questions as always if you found this useful please like comment share and subscribe and until the next one