The Power of Up-to-Date AI: Getting Started with Perplexity's Online LLM APIs

--- type: transcript date: 2023-11-29 youtube_id: LsZUOgvYhrY --- # Transcript: The Power of Up-to-Date AI: Getting Started with Perplexity's Online LLM APIs in this video I'm going to be showing you the new perplexity online llms that just came out so these are a first ofit kind llm API that allows you to have both fresh content as well as arguably content that has fewer hallucinations especially when you're quering about recent events where models like say GPT 3.5 or gp4 might have a lag and not have that underlying context of something like that happened either yesterday or a week ago or something along those lines so there's two models that just came out there's the 7B online model and then the 70b online model so the 7B online model is Created from the mistal 7B model which I'm sure a lot of you are aware of and Familiar of and then the uh perplexity 70b online model is trained from the Llama 70b model so there's an interesting blog post that I encourage you to look through and read through on all the different ins and outs of these models sort of the motivations behind them what they're helpful for and there is also some good examples of where it performs better than something like uh GPD 3.5 on a lot of metrics now I'm going to be showing you in the video how to set this up within your code so if you want to set this up with in a project you'll be able to do this by the end of the video I'll also show you where you can go ahead and play around with this in a playground if you want to just sort of try this out and see how well it works so the interesting thing with this is I actually asked uh in August if there was any um plans for offering an API so I'd been using perplexity for quite some time and I had the thought that it would be great to be able to access what they're using under the hood for those quick responses from their core offering and at the time and this was only in August they said they don't even have any apis planned at this time so fast forward a little bit to October that's when they first released released the perplexity API so this was at first uh released just to the pro members and it was released completely for free so there was some rate limits and things like that but it gave you access to a handful of models like the Llama models and the mistro models as well as a handful of others so the one thing to note with this and and sort of it goes without saying is these are all very new and there are some things to consider so for in from when they were first released just last month in October to now there are already some models that have been deprecated so I believe like the model from repet as well as I think the 13B variant of llama uh I don't see with in the documentation anymore so if you're looking to build something and use this say like the 13B model um I can't say how long it will be there I think it's moving out of beta so I think it's becoming a lot more stable where you you will be able to increasingly use these things more and more and any deprecations probably won't be as abrupt as as it was from last month to now now to actually go into perplexity just for a moment so if you're not familiar with perplexity it's uh a very simple interface and what it allows you to do is ask questions of things that are recent and it will give you that sort of chat GPT like feel but with a lot of extra bells and whistles so you see here that it has a little co-pilot that you can either turn on and off uh it has these sources that it's reaching for where it does some web scraping on the Fly and then it also has some images and follow-up questions so this is sort of the general um uh perplexity core offering here very powerful I use this every day I'm a huge fan of perplexity itself I'd encourage you if you uh have haven't tried perplexity try it out I also have a handful of videos on perplexity on my channel both building a product that's similar but then also just a general overview if you're interested in diving into it a little bit more so if we go uh just over here I'm just going to be showing you how to set up uh this from the API so the first thing that you can do is if you just go to docs. perplexity dodoc you will see this screen here now the first thing you have to do is they have the apis within your perplexity account so if you go over to the perplexity doai px- API this is going to be where you have to enable like you see here so you just have to set up your payment method and if you're already a subscribed member you can just say that you're going to use the same U billing information so right off the bat it might take a few minutes for your credits to show up so if you're a pro member you get $5 worth of credits every month that will refresh um that you're able to use you can also top up and all of that good stuff from here now once that's all set up you can go ahead and generate your API keys so I just have mine hidden here but you can copy them and you can generate multiple and whatnot so I'd expect this will probably grow and expand over time as they start to build out this offering some more so while you're on this page if you go ahead and just generate and copy one of the API Keys we'll just save that just for a moment to our clipboard and I'll show you how to get this all set up within vs code so another thing I want to point out to you is there some other things within the docs here so if we go over to guides on their docs and we just scroll down here or rather right limits so here's a list of all the different models that are available so there's mistl there's code llama there's llama 270b and then there's the perplexity 70b chat and then the online ones so today I'm just going to show you uh these models here but just know that if you want to leverage these other models they're also here as well so I'm just going to head over now and open up a completely new directory where we're going to get started with everything we need to get set up from their API and also while we're here I'm just going to pull up the API reference on the left hand side here so as you see here there's a number of different languages that you can use to set this up and it's also an interactive um uh set of documentation so if you actually toggle around some of the things here it will go ahead and put it within the example request which you can subsequently cop copy over to your editor so what I'm going to do is I'm going to be using bun in this example so I'm just going to say bun in it Dy so this will give me a a simple structure so it's sort of like mpm inet it will give me the typescript file and everything that I need now the first thing I'm going to do is I'm going to bun install API so I'm just going to bun API just like that and while I'm here I'm also going to touch. EnV So within your EnV so if you still have within your clipboard that API key that we copied you can just go ahead and put it in here just like this so I'm going to be using perplexity aior key and you can put in your API key just like that so don't forget to save that out and once that's saved out you can go ahead and close it so next we're going to go within our index.ts we can just get rid of this hello bun and the first thing that we're going to do is we're going to take a similar approach to this example here but we're just going to make a couple tweaks so the first line we're going to take is the line one here so while this looks like it could be something like an API key this is just an ID that's referencing the library for their SDK here so don't think that this is an API key and I'm exposing it here that's just referencing uh the actual SDK rapper to interact with their model itself so once we have that all set up what we can do is we can sdk. o now the thing that's nice with bun is we don't need to install something like EnV we can just reference that directly so if you have a EnV within your directory and you're using bun you don't need to have that additional package of using something like EnV so the first thing I'm going to do is I'm just going to uh move the messages above the actual query here like they have in the example so we're going to have our message content so there's our system prompt we're just going to say be uh precise and concise and then within our message we're going to say um this is a bit of a broken example but I'll just say what came out in the last version of Lang chain just to get a sense on uh what version is going to reference how dated it is and whatnot and then once we have that all we're going to do is we're just going to post to the chat completion and I'm going to start off with the 7B online version so once we have that we should be able to just go ahead and Bun index so one thing to note is as this is loading and I'll just pull this up here is there are some streaming options within here so if you'd like to set up streaming responses you can enable that you can pass it in just like you see here so you can stream true you'd have to just uh output those all those tokens bit by bit or have a mechanism to do so so depending on how you're going to be doing that or what you're going to be doing that for that's just a sort of an extra step here in this example I'm just going to keep it really really simple a handful of lines of code and show you how you can quickly get up and running with this but just know that that is an option within the offering here so uh back to our question so in the last version of L chain which is 0. uh 331 so if I just copy this over let's just paste it right in here so we see this was 7 days ago so already just from that query alone we know that it's reaching out to information that is relatively recent now the other thing thing that I did try is if I say something like what is the news today so I tried this example earlier and the thing that I found with this is what it actually gave back was similar to this where it's referencing almost the metadata of some of the top news sites and it's not actually telling me some of the top news but if I was more specific and I said what are the headlines on this particular news site uh potentially it could do better but I'm going to leave a lot of these things for sort of your exploration so if you want to toy around with it here you can go ahead set up this uh quick example I'll put all this code in the description of the video where you can just copy it over and get started or similarly just go within their docs pull down an uh an example and play around with there for whatever use case you're trying to set up so the other thing I wanted to point out before I close out the video is is the uh the link to labs. perplexity doai so this Labs is their interactive guey that you can access online and you can just try these models so if you don't want to try it within the code within the API you can just go ahead and try it within this uh labs. perplexity doai interface here so if I say uh tell me about the last golden date Warriors game and I query that you can see that it's getting information that was from yesterday you can see some information about the game and whatnot so just another Avenue this is something that I'm definitely going to be playing around with a lot more um just to see what is it good at what could it be better at um and all of those good things so that's pretty much it for this one the one thing I do want to point out is if you're interested in this type of content I have another video where I built out a llm web scraping API which is sort of similar to this um it's not definitely not as quick but it does offer some sort of General knowhow on how to set something like this up on the back end and I also have a perplexity clone that I buil with Lang chain and nextjs where it builds a perplexity like uh interface where you can go ahead and query something similar to their core offering so that's it for this one if you found this useful please like comment share and subscribe and otherwise until the next one