Groq API: Quick Guide with 5 Examples - Groq SDK, Langchain, LlamaIndex, OpenAI SDK, Vercel

--- type: transcript date: 2024-02-27 youtube_id: RbJBXcF3W80 --- # Transcript: Groq API: Quick Guide with 5 Examples - Groq SDK, Langchain, LlamaIndex, OpenAI SDK, Vercel in this video I'm going to be showing you how to get started with Gro which is absolutely by far the fastest inference API that's out there there's no one even close right now I'm going to be showing you five different examples on how you can get set up it goes without saying that these results are absolutely remarkable and I know people are incredibly excited to start building with this API by the end of the video I'll be showing you how to set up the versel AIS SDK to have a full stack nextjs application just like the example that I showed at the beginning of this video I'm going to be showing you how to get set up with all these different examples from scratch a little bit of housekeeping before I get started all of the links that you see within the video I'm going to put with in the description of the video which you can check out I'm also going to put a GitHub repository where you can pull down and have all of these examples on if you want to play around with them or have some boiler plate where you can quickly reach for them with all these different ways you can interact with Grog one thing to note I am going to be running bun with a number of commands this isn't required you can substitute this for something like npm or pnpm or yarn depending on what you're setting up but if you don't have bun installed I encourage you to check it out it's a really great JavaScript and typescript runtime as well as toolkit that's designed to speed things up so if you're interested in installing bun and following along just make sure that you do have bun installed all right so the first thing that you're going to have to do for all these examples is you're going to have to make an account at console. gro.com so grock with a Q always remember grock with a Q I am definitely still getting used to this when I'm setting up some of the environment variables I definitely put in Gro with a K nevertheless just keep remembering grock with a Q grock with a Q just as a sign note did come before grock with a K just putting it out there once you're logged in you have a playground that you can play around with just like you would the open a API so you have the area for your system prompt your user messages and then you also have a couple different models that you can select from but time of recording there's the Lama 270b model which accepts 496 tokens of contacts and then there's mistal ai's mixl 8X 7B model which allows for over 32,000 tokens of contacts on input so what we're going to do is we're going to open open up a brand new folder within vs codes you see we have nothing within this folder here the first thing that we're going to do is we're going to go within the code editor your choice we're going to set up five different directories here I'll just show you from the vs code guey so for our first directory we're going to call it one Das gr- SDK then our second directory we're going to call it 2-op a SDK for our third directory we're going to call it 3- Lang chain for our fourth we're going to call it four- Lama index X fth we're going to call it 5- forell AI SDK now that we have all the directories started we're going to CD within our first directory here so the first thing that we're do is we're just going to mpm install the groc SDK we're going to touch index JS to have an entry point for our application and then I'm just going to bring down the terminal here so we can see all of our code then what you can do here is you can just copy this first half here put it within your terminal we can do here is we can just take this we can go over to our terminal and then I'm not going to be showing this on screen but you can just go ahead click API keys and then put the second half we have our project installed we have the SDK installed we can go ahead and run this so if we just node index.js we have a response back from the Mix end points if you wanted to swap this out for the Llama model take the API string here swap it in within the model here and then you can go ahead and generate another response and this time you'll be getting it from the llam to be model now the other nice thing is because it conforms to the open AI schema you can also use the open AI SDK as well so say if you're already using something like GPD 3.5 or gd4 and you want to swap in something like the mixer model and get that inference speed that Gro offers I'll be showing you how to do that in the next example here to go within our opening istd what we're going to CD out of this folder and then we're going to CD into open AI here similar to the first example we have a completely empty directory this example I'm going to be showing you how to get started with bun so if I just bun n-y this will spin up a simple bun project where you can get started you have your index TS all of the different things that you need set up you even have a g ignore you're ready to go and get installing things we're going to bun install openai and then we're going to head over to platform. open.com and go to one of their quick start examples in this case we're just going to copy the chat completion but we're going to be making a few different tweaks so we're going to copy that within here with the open AI SDK version 4 it assumes that it's going to reach for the openai API key within your process so if you have it already within the process it's going to look for that variable but in this case we're going to override a couple things here we're going to use the API key of Gro instead of open AI we're going to be swapping that out within our environment variable and using this value here and the other thing that we're going to have to put Within here is the base URL so essentially what this base URL is doing is just where we're routing that request so in this case instead of routing it to the open a API we're just going to take that over and we're going to send the request to The Croc API instead now the last thing that we have to swap out is just the model So within the chat completion here like you see here we can just go ahead and replace the model with the one that we want to use here I'll just use the mixt model for demonstration once we have that set up we're going to create a EnV and within the EnV the value that we're going to take is this groc API key we're going to put that here we're going to put an equal sign and then just like we had in our other example we're going to grab an API key we can use the one from the previous example or generate another one close out your EnV file just make sure you save it and everything and then similarly with your index TS just make sure you've saved that out and then the nice thing with bun is you don't need to import something like EnV so if you are using something like an older version of node.js you will have to install a package called EnV to go along with this example and then you'll have Tov doc config but if you're running bun just like I'm showing you you can just go ahead and run index and it will find those environment variables that you have so if I just bun index.ts and there you have it you have the response back from the gro mpoint using the opening SDK so with just three lines of code that API key the base URL and then the model you can pivot to something like the gro inference API in the next example we're going to CD out and then CD into three- Lang chain now for the L chain example just like we did in the opening istk we're going to bun init Dy the L chain documentation we are going to be installing the Lang chain Croc package and Lang chain itself so want to install Lang chain and then Lang chain groc once we have those set up we can open our index TS then we just copy over an example like they have in their documentation we can put that here just clean it up a little bit get rid of the example and then just like we did within the opening istk copy your environment variable so go ahead and copy that over make sure you paste it to the root of the directory so long as you have it just like that they already have it set up with the groor API key and then similarly you can just go ahead and Bun index and then there you have it all set up within your Lang chain project with the different schema all set up with the schema and tooling that Lang chain enables next we're going to go ahead CD out we're going to close this and then we're going to go within our llama index example and you can probably guess by now what we're going to do we're going to bun init Dy we're going to head on over to the Llama index documentation what we're going to do here is we're going to grab the the full example the one thing that's different with this llama index example is this is a simple rag application what we're going to do in this example is essentially we're going to load up a file so you could swap this out for any file that you want we're going to read that file we're going to set up a simple document which has some metadata that we're going to pass in to our Vector storage then from here we're going to create the vectors vectors are essentially a numerical representation on how similar different things are to one another the easy way to think about vectors is the closer together that numerical representation of the different items the more similar that they are the further way those numbers are is the more unrelated now the one thing to note with llama index is it does default within this example to using the openai API for embedding So within ourv we're going to have two different environment variables within ourv we're going to put that grock API key also have to go to platform. open.com and generate an openai API key once we have our DV saved out and the index TS we can just go ahead and Bun install llama index and then we can just go ahead and run this what this is doing is we're going to have this query of what is the meaning of life and it's going to reference this text file here based on the context from the text file this is the answer that is generated for us if you're interested in building out a reg application this is just one Avenue on how you can get started with a llama index in the last example I'm going to be showing you the versel AIS SDK so if you haven't used the versel AIS SDK it's a really great place where you can go ahead and get a lot of different boilerplate options for a ton of different offerings so from anthropic to fireworks to cohere to mistol replicate perplexity the list goes on there's a ton of different next example next examples felt kit solid start a ton of different JavaScript Frameworks as well as llm and inference apis that you can interact with with these different examples so in this example I'm going to be showing you next open Ai and what we're going to be doing is something similar to the example with the open a SDK so what you can do is grab the script from here you can choose npx yarn or pmpm you could also run this with bunx if you'd like we're going to get out of this directory just like we did the others we're going to go within our verel directory to keep everything organized and then within our versel AI directory I'm going to run this command which should just take a moment once that's all installed we're going to go within the directory of our apps so the next opening ey app we're going to go within that directory the only thing that we're going to swap out is within the API chat route So within here like I mentioned very similar to the opening isk what we're going to do is we're essentially going to override the openi API CLI here so we're going to replace all of this and then just like we had in the other one we're going to replace the API key with our grock API key and then for the base URL for our endpoint we're going to route that to the gro API and then all that we have to do is just swap out the model just like we did in the other example once we've done that we're going to just back out of our directories here that we've edited then within our env. loal we're going to get rid of where it says example So within the env. loal we can just get rid of that openi API key and replace it with the same API key we've been using through all of the different examples to get started all we have to do is mpm randev and then open up your environment you'll have that simple nextjs application the outside of the video If I say tell me 200 things that a CEO should list in five words per bullet and then if I just run that here we see how fast that this runs so it's incredibly fast this was a subscriber or a viewer on my channel that gave me this idea on how to test one of these really fast inference apis on just listing out 200 things that a CEO should know and I think this is a great example to show you how blazingly fast their inference API is so kudos to the team at Gro and what they're building it's absolutely incredible I'm excited to show you a ton of different examples on this channel on how you can get started on leveraging this grock inference API by leveraging the lpu chips we're going to be able to build some pretty incredible llm AI applications with this so if you want to see different examples on different projects and how you can build out different things whether it's voice applications or different rag applications or clones of different products stay tuned to the channel but otherwise that's it for this video if you found this video useful please like comment share and subscribe otherwise until the next