Langchain and Pinecone with Node.js: A Quick Start Tutorial

--- type: transcript date: 2023-05-11 youtube_id: CF5buEVrYwo --- # Transcript: Langchain and Pinecone with Node.js: A Quick Start Tutorial in this video I'm going to show you how to get started with line chain and Pinecone in node.js we'll set up an application where it will run through a directory of documents like we have here and we'll take those documents we'll break those documents up we'll query the openai embeddings API once that's returned we'll upload those embeddings as a vectors into our database then finally once they're in our database we'll go ahead query the database get the top results and then pass it to an llm for an answer so we'll be able to ask a question and then get a result based on what we have uploaded in our database and what documents we have used so all that will need to get started with this are our API keys we're not going to be setting up anything in the GUI on on Pinecone we're going to do everything from scratch in node.js so the way that I have this structured is we have four main files that we're going to be working out of I have them numbered in both the order that we're going to be using the them and going through them in this video but also the order that they're going to be used in the final state of the application so we have our entry point our main.js this is going to be our actual uh node.js file that we run and then from there we're going to have some files that we've exported into our main main.js so when to create our index one to update our pine cone instance with our our vectors and embed and then we're going to query Pinecone and query the GPT model to get a result so the first thing that I'm going to have you do off the bat is we're going to go npm init Dash y so that will get our project all set up to install some dependencies so once you've done that you should have a package Json on the the left here in your vs code or code editor then once we have that set up go ahead and make these four files here so we're going to have a main create pine cone Etc so once you have those set up go ahead and create a DOT EnV so in our DOT EnV once you have that set up you can sort of go split screen if you'd like or just hop back and forth we're going to go over to the openai API page so you can go platform.openai.com account slash API Keys create a new key and then once you have that key just paste it in here so one thing to note with the openai API you do have some free credits where you don't need to tie in a credit card off the bat so if you want to try this and you don't want to integrate your credit card into their system you don't you don't need to worry about that you should have more than enough credits to get off the ground and running now similar with Pinecone you do get a free index so there are some limitations with the free account but you will be able to do everything that I'm showing you without tying in a credit card as well so once you've set up the openai API key in your dot EnV make an account on Pinecone if you haven't already then just go to the API key section so here we're going to need both the environment and the value of the API key so the environment goes just as you see here and then the API key goes here so once we have that I'm just going to hop back to our mean JS with our instructions here we're going to install a handful of things so while I do that I actually want to pull up the package Json so in the package Json we have four dependencies we have the pine cone dependency dot EnV Lang chain and PDF pars so I just want to touch on PDF parts for for a moment so if you're using PDF so you'll see in this example I'm not using PDFs but by the end of this video you'll be able to incorporate PDFs to read and store if you'd like but if you'd like to include other documents like docx or ePub there's going to be some other dependencies that you're going to have to reach for so I'll cover that in just a moment but just one thing to note you may depending on on what you're trying to build need to install some other things as well so I also want to touch on the version so I'm running node version 19 and if you're within this range of between 17 and under 20 let's say you should be good to get this all set up so if you npmi all of these things here pinecone.mv line chain PDF Parts you should be good to go so just make sure you're not running like node version 14 or something where a lot of the the dependencies might fall out of date and this might not work so you can check your node version by just typing in node-v in the terminal and you'll see if you follow within that criteria so once we have that all set up and installed we've obtained our API Keys we've put them in the dot EnV make sure you save that dot EnV just so we don't forget later on when we go to run and like I noted earlier is so for the file loaders so we're going to have to specify them one by one and they could use different dependencies like I mentioned so if you want to see what kind of file loaders that there are you can just head over to this link and take a look so csv.x Etc so if you go to docx you'll see you'll need this dependency and there's others for a particular file loaders so ePub that you'll need these two dependencies and whatnot so I'll touch on this again you'll need to import those and install them if you need them and then we'll we'll reference them in the loader in just a moment here where I'll show you so the first thing that we're going to do is we're going to require a handful of things so we're going to require the pine cone client itself so this is going to be how we actually create the index from the back end and then also update and query so it's pretty integral to what we're doing here is the pine cone dependency now the directory loader this is the way that we can actually read everything that's within this directory and then for each file type you'll have to specify them here so if you had the Epub loader just import the Epub loader from you know the proper spot whatnot so we're going to require the dot EnV that sort of self-explanatory that's how we reach into our DOT EnV and read the API keys that will need to interact with both open Ai and Pi coin pine cone rather so then here we're going to import the files that we're going to be building out after this one so we have our create pine cone index the up update pine cone and query pine cone and query GPT so we're going to initialize our DOT EnV and then from there we're going to initialize our loader so this is where we'll set up all the different file paths or file types and pass that we're going to be setting up so if you had ePub you could just add a new line here ePub make sure you have the loader and installed and whatnot and then just you know new ePub loader and whatnot then from there we're going to actually call this and it will reach return an array of all the documents with some metadata so from there we're going to set up some variables for how we're actually going to interact with this so I have a couple documents here so I have Romeo and Juliet and The Great Gatsby so you can go to if you just Google the Gutenberg project there's a handful of books that you can just grab and use if you need something as an example otherwise this should work with any other text files or other file types if you include the loaders that you need so the index name that's going to be the actual pine cone index so if you've made one already you can go ahead and put that within this line here otherwise once we call this for the first time we'll just initialize it with the name that we pass here so the vector Dimension so since we're using the open AI embeddings model and endpoint this is the criteria that's returned from their embeddings endpoint so if you're using a different one or want to use a different one just know you will get you will need to set up the proper Vector dimension for the database so the next thing that we're going to do is we're going to initialize our pine cone client and pass in our API key and the environment so those were those things that we just reached for in the the GUI for Pinecone so once we've done that we're going to go ahead and run our main asynchronous function so in the asynchronous function we're going to check if Pinecone exists and then we're going to create it if necessary so the first time around that's going to be where it actually creates it otherwise it's just essentially going to fall in the other condition of this of this file here and then just skip on to the the next function then next in this one we're going to actually update our pine cone so and you see within all of these I'll just pull them all up here though the one thing in common we're going to be passing in our client into all of them and then also our index name but in each of them we have some unique things that we're going to be passing in so for the first one we're going to pass in our Vector Dimension we won't need it in any of these our second one we're going to be passing in the docs that's going to be where we Loop through the docs break them up into chunks embed them then store them as vectors and then finally when we query them we're going to pass in the question that we need so in the create pine cone index you'll see how we're sort of going to structure all these files so we're going to export const create Pinecone index and then we're going to pass in the parameters that we need for it so first we're going to see if the index exists so we can do that with this client list index and then if it doesn't exist that's going to be where we actually go ahead and create the index so this will run the first time and once it's run it will just skip to its the the else condition so one thing to note with this as well is I do have this await new promise set resolve to 60 seconds so you can you could potentially remove this so it's it could take a bit longer I've sort of noticed it takes about a minute or so to set up that initial index and once it's set up um so say you run it and run into an error just run it again once it's actually initialized so you can see it in the GUI once it's set up and whatnot but this will just sort of um you know you could bump this up if you wanted so once we have that once it's all set up and running we can go ahead and log it for when we're running we're running through either updating or querying all right next we're going to be setting up the update Pinecone function so this is going to be the most involved function that we're writing in terms of the the logic that we're going to be writing ourselves so the first thing that we're going to do is we're going to import a couple required modules so we're going to import the open AI embeddings module as well as the recursive character text splatter splitter so the open AI embeddings endpoint so we're going to use text Ada embeddings 2 to actually create embeddings of the text chunks and then to create those text chunks we're going to use this recursive character text splitter so similar to our last function we're going to have that same structure export const update Pinecone and then we're going to be logging out an awful lot here and the reason for that is hopefully to a have you understand what's going on here and then B if there's any errors that come up you'll be able to hopefully quickly go in and debug or you know reference the area where an issue is coming up so we're going to retrieve the Pinecone index so I'll do so by calling the dot index method and passing in the index name and then from there we're going to just log out the index that we've retrieved so in this we're going to create a for Loop because the docs that we're passing within the update Pinecone function is an array so we're going to run through all the different docs one by one so and part of the reason why we have a for Loop we don't have like a fancy es6 you know for each or something like that is we want to be able to reference it within the context of the asynchronous function so you can do that you can await by using a for Loop in in JavaScript like this essentially so you know as much as I love to use something like 4-H you know that's sort of why we have it set up the way that we do so once we have that we're going to Cache a couple variables that we're just going to use in a second here and then we're also going to log out the file name so here we're going to create the instance of our character splitter so there's a bit of a debate on the optimal chunk size so you know some people might say oh 200 is good because when it scans through it will give you like you know a a you know a range within like say like a sentence or two or within that realm if you do a chunk size of a thousand that's the one that I've sort of uh gotten used to and comfortable with now if you push the limits and really go to um you know how many characters it can handle it might not work as well because you're you're essentially having it go through these big blocks of text and then you could potentially run into token limits when when or if you want to pass it to an llm so you can play around with that I'm going to use a thousand in this example but just know there's a sort of an interesting piece there you can look into if you'd like to um and play around with so then we'll log out we're splitting it in chunks and then we're going to Simply pass in our text so our text is going to be the uh the page content from the docs object of of each of them and then this will break it up into chunks so once it's broken out into chunks we'll log out how many chunks that we have that we're dealing with that will give us hence on how long it will take and whatnot and just we you know it's sort of useful information to have so once we've done that we're going to actually embed the documents so I'm going to remove some new lines here and then we're just going to go through one by one and embed all of them then we'll log out when they're they're done embedding and then just to confirm we'll show we have the length of the embedding so there should be continuity between these numbers then finally we're going to create a batch of a hundred uh vectors that we're going to send to Pinecone now the reason we have 100 is that's what pine cone specifies as the optimal number to upload just as an aside so if I went to upload The Great Gatsby and embed all that I'd run into an issue where it's above that two megabyte threshold that where that it won't query and upsert that into the database so you need to batch them you know it's sort of this is the happy medium between uploading vectors one by one and you know worrying about running into that that two megabyte limit and is what they said is sort of the optimal number so that's why we have it set up here so it looks like a a lot at first glance here but really the things to um to consider here is what we're going to be passing into our Vector so we're going to be creating an ID we're going to be passing in our actual Vector so this is going to be the numerical representation of that piece of text that we get from openai and then we're going to create our metadata so we'll have the location of where it shows in the the text file in this case we're going to have the page content that we're going to pass in so instead of referencing it within a separate SQL database like you potentially could with a Pinecone setup we're going to just have that text embedded in the metadata as well and then finally we're going to have the text path where you can sort of use this if you want say you want to have a reference where you could say this these answers came from these files or websites or whatever you so then we're just going to actually Loop through and batch them and then we're going to upsert them this is going to be how we actually get them into Pinecone so once we've done that we're going to say okay pine cone index is updated with this many chunks and that's it for this file all right in this last file what we're going to be doing is we're going to be querying the pine cone index and then we're going to be taking those top results and then passing it into an llm with our question so the first thing that we're going to do is we're going to require a handful of things so we're going to require the open AI embedding exam point so this is going to be what we use to in not in code but in bed rather our question so that's going to be essentially what Pinecone runs through on the back end it's going to say okay we have this Vector now run through the index and see which ones have the highest relatedness between them so once we've done that we're going to import open AI this is going to be how we actually use the llm that we'd like then we're going to load QA stuff chain so this is one of the chains available in line chain there's a handful of them I expect some of these to change over time and more to be added but essentially what this is doing is taking the context of our results and the question and then having that question be asked on the context so there's some sort of nuances there within all the different um chains that do a similar thing so you can play around with these if you'd like you can go to their documentation and look at chains and then we're simply going to require document from blank chain so here similar to our other files we're going to export const function name taken our parameters then we're going to log out that we're within this function and sort of walk through the steps of what we're doing like we've done in the other files so we're going to retrieve the pine cone index like we did in the other files then we're going to query the open AI embeddings endpoint and we're going to pass it in the question so once we've done that we're going to specify how our query for the pine cone index so in this one the top K that's the number of results that we want to have returned if there's less than 10 say if there's not that many matches it will just return less than 10 it could return 0 obviously if save you you know query something to do with like what's the best MacBook in Romeo and Juliet and The Great Gatsby it should return you zero then we're going to pass in our Vector here then we're going to ask it to return that metadata so like we had in our update pine cone So within the metadata we have some things that we could use like that the reference of where it came from but in this case we're just going to be using the actual text content that we we included in our metadata so then we're going to log out if it's found matches so we're going to say okay we found 10 matches let's say and then we're going to say okay now we're asking our question and just log out that question whatever we had in that main file so once we've done that we're going to go ahead and set up a two separate conditions so if there's matches go ahead uh set up that instance of openai load in the chain and pass in our llm into the chain and then we're going to extract extract and concatenate our 10 results or however many get returned then we're going to execute the chain so we're going to pass in our question and then our input documents here and then finally we're going to log out the answer if we have one otherwise we'll say since there are no matches GPT 3 will will not be queried so you can use gpt4 if you want I'm using gpt3 for this just because I I found it has worked well enough and you know the cost is considerably cheaper but feel free to use whatever you'd like so that's pretty much it so now we should be able to save and run the application so if I say node and let it run we can see okay it's checking it's splitting it up so since mine's already been embedded and whatnot I'm just going to go in and show you how you could potentially just play around with this or set up separate conditions if you'd like you know not to have it updated you know you can maybe have like a manifest or or a you know a more sophisticated way of checking whether they those files have already been uploaded into Pinecone but just to show you how this works so if I just um turn off that update pine cone which is arguably the the most involved and function that will take the most time especially if you have longer documents and if I just go ahead and run it we'll see that the Pinecone instance has been created but then it will essentially just go ahead and ask our results so it will embed that question and then once it's embedded that question it will go ahead and give us the result so it's asking the question we see here it found 10 matches now we're just waiting for gpt3 to return the results and it should only take a moment here so we see Mr Gatsby is etc etc so that's pretty much it for this video if you found this useful please like comment share and subscribe I have a lot more link chain content and things around this sort of AI boom that we see occurring right now coming out so yeah if you found this useful or like this content I look forward to seeing you in the next one so until then