Upstash Serverless Vector Database and OpenAI Text Embedding 3 in Node.JS

--- type: transcript date: 2024-02-02 youtube_id: 4S5HTsOWI38 --- # Transcript: Upstash Serverless Vector Database and OpenAI Text Embedding 3 in Node.JS in this video I'm going to be showing you how to get started with up Stash's serverless Vector database that just came out I'm going to be also showing you how to get started with the new open AI embeddings 3 large model and I'm going to be setting up everything within nodejs and I'm going to be showing you how easy it is to set up a vector database on up stash the first thing that you're going to have to do is make an account for up stash they have a generous free tier for the different services that they offer once you're logged in if you go over to Vector you can create a new index you can name it whatever You' like we have a options in terms of regions that you can choose from you can choose Ireland or you can choose Virginia I'm going to be showing you how to set it up with 256 Dimensions one of the reasons I want to set it to 256 Dimensions is I want to demonstrate how you can use shortening embeddings from the new text embeddings endpoint from open AI it's a really neat feature where you can have up to 372 Dimensions or as small as 256 Dimensions now just a few quick tips on how to choose the size of Dimensions if you're considering this so on the higher end of the spectrum if if you're looking to use the full-fledged 372 dimensions that's going to cost more but it's going to be more accurate one thing I haven't tested out but it is a bit of an assumption is I'd imagine the smaller the dimension size the faster the retrieval is going to be from your vector database the first thing that you're going to do is just open up a new workspace within VSS codes if you have an empty directory you can npm in it- y from there once initialized we're going to install a couple dependencies so we're going to install the up stash Vector dependency as well as AI once you have that you can go ahead and touch index JS as well ASV within your EnV we're going to get a few different environment variables to get your open a key just log into your openai account grab the API key and paste it in just like this then for up stash once you set up your vector database if I just go ahead and set this up you can see we're going to choose the free option in this example and you're able to go over to the EnV here and you're able to copy these and paste them in now that we have that set up we're going to head with in our index.js we're going to have three dependencies we're going to have open AI we're going to have up stash Vector as well ASV for our environment variables what we're going to do is we're going to create an open AI instance for our API calls you don't need to explicitly pass in your API key so long as you follow this format so the open aore aior key within their V4 release of their node SDK you no longer have to explicitly set that from here we're going to initialize our new up stash Vector index with our environment variables that we just paste it over next what we're going to do is we're going to create a number of different functions the first function that we're going to create is a function to generate the embeddings for a given text we're going to wrap everything within a tri catch just so we know if anything fails within our application within our Tri catch is we're going to go ahead and create embeddings so we're going to specify that the model is text embeddings three large we're going to pass in the text that we're going to be passing in near the end of the application here and then this is going to be how we set the dimensions that we want return from the opening eye what embeddings are if you're not familiar is essentially they're a numerical representation of what you're sending at so if you're sending at text it's going to be the relatedness between different items if you think about it in terms of zoo animals if you think about an elephant a lion and a tiger zoo animals just as an example are going to be numerically grouped together but if you go ahead and embed something like a Toyota or a different car that's going to be grouped in a different position those floating Point numbers within the array that gets returned are going to be f away if they're less related to the vectors it's comparing it to so here we're just going to see whether we get a valid response from the open AI endpoint if we do we're going to return that result and if we don't we're just going to return that we got an unexpected structure and an error had occurred then we're going to log out the error if this fails next we're going to have a function that splits our text and upserts the data within our up stash Vector database you can use something like Lang chain or llama index to split up all your text if you you'd like but it's pretty simple to set something like this straight within the JavaScript within your code we're going to establish the chunk so you don't need to worry about the reject too much but essentially what this is doing is it's going to be chunking based on a thousand characters within the text that is passed in we're not going to allow anything higher than 1,000 characters to be sent to the open AI embeddings API then from there we're going to Loop through all of the different chunks we're going to give a chunk ID to each of them then with each chunk of text we're going to go ahead and generate those embeddings once we have those embeddings returned we're going to be inserting them into our Vector database with an up stash so we're going to be sending in a handful of things we're going to be sending in an ID we're going to be sending in the vector itself and then we're going to be sending in the metadata of the text itself that was embedded now within the metadata you can include a lot of different things so say if there's a URL or a file path other information that might be relevant you can throw that all within the metadata here then we're just going to console log and next what we're going to do is we're going to establish a function that is going to query that vector database now to query a vector database you also have to pass in that query as an embedding you can't just pass in raw text so when you send in a query it's going to embed that you're going to have that numerical representation and that's going to be what it uses to compare within the vector database so it's going to scan through all of those different vectors and find the ones that are most related within the parameters of the function we're also going to allow for the default of up to the top three results to be returned back the first thing we're going to do is we're going to go ahead and generate that embedding from there we're going to go ahead and query our index we're going to be passing in our query embedding we're going to be passing in the number of results that we want and then we're going to pass in that we do want that metadata returned then from there we're going to log out the result alternatively if there are any errors we're going to log those out and throw an error the last function we're going to write before our main function that's going to demonstrate all of these different functions and how you can use them within an application is going to be a way on how you can interact with an llm so a popular application for vector databases is rag or retrieval augmented Generation Now what we can do with this is we can pass in our top results to the llm with some simple instructions I'm going to be showing you just a very basic example on how you can set something up like this without having to use a framework or anything like that we're going to do is we're going to leverage that function we just wrote we're going to go ahead and ask for the top three results for the query that we're sending in then what we're going to do once we get those top results back we're going to log them out and just so we see everything is coming back as we expect then from there we're going to set up a chat completion from open AI within our chat completion we're just going to declare messages and our system message we're going to have it be you are an expert retrieval augmented generation system respond to the query with the information from the top result text so you can really finesse this a bit more if you'd like but this is just like I mentioned basic example then what we're going to do is we're going to send it in the query so you could imagine a chat application where a user asks a question I'm just going to stringify all the top results now you can normalize the data if you want you don't necessarily need to pass in the things like the ID or the score things like that but in this case I'm just going to show you a really quick example on how you could do this so I'm just going to specify that we're going to be using GPT 3.5 turbo because it is a relatively cheap option once we've added those vectors to our database we're going to go ahead and query them we're going to perform that similarity search we're going to get the top results from those and then this is going to be how we send those results to something like GPT 3.5 or gp4 to essentially summarize and give us the answer that we're looking for what I'm going to do is I'm first just going to run the top functions here so I'm going to run these one by one just so you see how exactly they all work if I go ahead and node index JS we'll see that the records with the ID and the text were upsert and what you can do here within up stash which is really nice you can go over to the data browser and you can see both the vectors as well as the metadata you can also pass in a query here if you'd like to go ahead and search the vectors within the guey it was a really nice option to be able to interact with it within the guey I'm going to go ahead and comment these out now that we know we have some data within Vector database and then I'm going to go ahead and send a query so our query is which Sue animals can dive the deepest we're going to go ahead and run this if I pull up our console here we see our top results so we see the top three results we see that we have our elevant Vector we have our penguin vector and our giraffe vector and we have a score we see that the text for the elephant ID and the Elephant text is higher than the Penguin and giraffe text now that means that it is more similar now just to show you the last example if I just go ahead and comment out query by text and send the results to open AI remember within this function we still use Query by text all of that text that you just saw there we're just sending that Json stringified object to open AI like I mentioned you could normalize that data pass in fewer tokens if you'd like or only pass in the metadata that you'd like within the llm and if I just save this and run this one last time here so you'll see that our result back from open AI is the zoo animal that can die the deepest is the elephant elephants are known to be excellent swimmers and can dive up to 30 ft underwater which is referencing pretty much exactly the information that we're passing in within our Vector database this is just a quick example on how you can use both up stash as well as text embeddings 3 from open AI if you found this video useful please like comment share and subscribe and otherwise until the next one