Google's Vertex AI in Node.js

--- type: transcript date: 2023-06-12 youtube_id: x5o8DVj6QD4 --- # Transcript: Google's Vertex AI in Node.js in this video I'm going to be showing you how to get set up with the Google's Foundation models within model garden and vertex AI on Google Cloud so a prerequisite for this as you might imagine is you will need a Google Cloud account if you don't already have one don't worry because when you sign up there is a likelihood or possibility at least that you will be able to have some credit so I just signed up last week for this and I got a handful of credits that I'm going to be able to use on their platform so once you're set up you'll have your list of projects here that you can go ahead you can make a new project and the one thing to note is if you're using my first project or for whatever project you set up for that matter just make sure you go to their dashboard and make sure you enable the apis that you're going to be using so as you see here I have the vertex AI API set up Google storage API you can enable all recommended apis might be the easiest if you plan on using a handful of these things but just make sure you go ahead and enable those things if you don't do that they won't work so next we're going to install the gcloud CLI so the reason for this is the way that I have this set up is we're going to be reaching for their credentials globally within our system so in other words any project that we have with this setup you will be able to reach for those credentials and it work now if you want to uh you know make environment variables or change the authentication change the authentication method later you can go ahead and do so but I'm just sort of showing you a way for ease of use so once you come here you just select your system run through the steps I obviously won't go through all of these there's a slight different steps for all different systems and different packages that you might need to install so just find what matches your system make sure Python's installed download the package and just walk through the steps here so once you have that installed you can go ahead and authenticate an application and have the default authentication across your system so you can just go gcloud auth Application default log in then you'll have a browser window pop up log in like you would to Google then once that's successful you'll have that saved locally and hopefully you won't have to worry about authentication going forward so once you have that set up and you've made sure to enable the apis here you can go to the model Garden again and you can start to take a look at the different models that are available so there's quite a few that are available there's some that still aren't available so most notably they're they're unicorn models so they're sort of a spectrum from gecko all the way up to unicorn with bison right before unicorn which is what I'm going to be showing you today so if we go into bison for instance you just go down here you can sort of see some of the steps to actually set this up so the thing that I noticed with their documentation right now is if we open up the prompt Design This is similar to the playground with an open AI if you go and view the code you don't actually have a node.js example here so hopefully something that's coming down the pike but for those people that work with a node.js JavaScript developers I'm going to show you the wrapper that I made and some examples on interacting with a few of their different models that I'm using so you can go in here and use this if you're a python developer and you'll have all your the things that you need you'll have your project ID the model and the example prompt if you have something within there and then similarly you can have the curl request so I'm actually going to have this open just for a second to show you where you can plug in some of the details that you'll need in some of the examples so the way that I've set this up is I have this parameters object that we are sending to our send request method which I'll go through in just a moment but the things that we'll need once we're authenticated is we'll need our project ID which you can see here we'll need the API endpoint which you see here and then you'll need the model ID now the other thing with the model ID is or below the model ID is the instances so there is a slightly different structure for each of them so just one thing to note is if you're going through and you know trying out a bunch of different models there is a different schema that you might have to use so once that's all set up and you have this as an example all the examples that I have are going to use these parameters you could further scaffold this out and you know include a DOT EnV here and use environment variables if you wanted but for now I'm just going to be showing you how to pass these in one by one so the first thing I'm going to do is I'm going to go to our Google Foundation models JS so the only dependency that we're going to be using and I'll just make this a little bigger here and increase the size of everything is the one thing that we're going to be requiring here within our project is the Google auth library so if you reach for GitHub I'll have all this code here you can just npm install to get the dependency that you need or if you're following along within the video and your npm and it Dash Y and doing it sort of one by one as I show you just make sure to install the Google auth dependency so the first thing I'll show you is the utility function so the main function really is just going to be the send request these are just sort of help helper functions that you don't actually need for the core logic of interacting with these models so the first thing we'll do we'll install our dependencies or require dependencies rather Google auth fs and path now this is what I've used for our wrapper here so first we're going to be setting up our authentication reaching for our access token so with it in the library it has you know the the it knows to reach first for your local version of all the credentials and then from there it's going to get an access token and then that access token is what we're going to be using in our fetch request so we're sort of creating our fetch request here which we're going to use for all our different models here and then we're going to be passing in that parameters object which has the things like our prompt and other settings that we might want to play with so it's going to be a post request this is going to be where we pass in our access token and then finally the parameters that we have passed in into the function we're just going to stringify them for the request so we'll handle any errors here if they come up and then if there aren't any errors we'll re return that response back to our function where we'll console log it out and do a couple other things so next I'll just quickly run through write response locally all this is doing you don't really need to pay too much attention to it is we're going to be writing the response that we get back from the API locally to our system so just to keep this video simple I'm just going to be showing you you know what the output looks like and I've found it local just or helpful rather to save them locally look at the schema see what's there see what you can work with while you're starting to play around with these models and similar to the function Above This one is just going to be for saving code files so if it detects that there's a piece of code say jsx it's going to go in and take that jsx use it as the file extension and then use the contents and save it out as a standalone file for their their coding model that I'm going to be showing you and that's basically it so then we're just going to be exporting those to be able to use them in our next examples so the first one that we're going to be using the model is their text bison model so like I mentioned this is like the one below their largest model at least at present so within their Palm to Umbrella there's four models from gecko to unicorn and the unicorn model I did try it isn't available even though it's not listed there yet so hopefully coming soon and this is you know what's available to us for now that we can use so pretty straightforward we have our API endpoint we have the project ID where we have this all set up we have the model ID we have the instances array that we're going to be passing in and in this case we're going to be passing in the content which is essentially like our prompt our query we have our parameters so you can set the temperature so our creative you want it to be the output and a handful of other things so what we're going to be able to do here is once you have that all set up you can just go ahead within your examples folder make sure you're in that folder you can just go node one text bison we can wait a moment for a response and then if we look here in our example output we'll see where that output um file is saved and then you can go ahead and look at the response so if you don't want to look at things within the terminal you can just look here so you can see okay you have this citation metadata I have seen so GitHub links often appear within the citations here safety attributes and whatnot so I'm not sure exactly what all of these things are but I did see you can dial up or down this safety of the responses so if that's something that's important to you you can look into that and then the other thing to note with the response is as sort of general knowledge with llms is they're not 100 accurate so in this one no citation and it's actually wrong so it wasn't version 10 but that's sort of a side point you know this model can do quite a few things but just a reminder that you know these things aren't absolute and you know if you're relying on it for facts it's don't at least not not yet so but the nice thing is is there are citation metadata so there might be a plan if you plan on using this um you know four things that are potentially facts um you'll be able to have the citation metadata so I haven't seen this come up with most of my responses but I have seen it come up with a handful so far so definitely something nice to see so from there I'm going to hop into the code bison model so a lot of these are very similar um this one we're going to be requiring the save to code or save code to file for what I had just run through so if it's jsx or HTML for your query so in this one you'll see that the instances array that we're passing in is a little bit different so we're using prefix instead of content here so there's a little slight nuances there so you can swap out your project ID if you have different you know configurations for each of them and otherwise you know you can plug this into like an environment variable if you don't want to pass these things in here and make a little tweak to the code but so in this one I'll just say okay write me an encapsulated leaflet focused on San Francisco in a single jsx file so leaflet is a mapping Library where you'll be you use a simple sort of you can think of it as like a Google Maps an open source Google Maps so I'm asking it to return me that within jsx and with our little helper function here we'll just wait for a moment it's a little bit more code or of a response than our last example but it's still pretty quick and you can see here so it saved out our response we can first take a look at that or rather this one and you can see okay so the contents there it's stripping out all of this stuff so the GSX and then it's nicely putting it within a file here that we can use so sort of a helpful helpful little function that you might find helpful depending on your use case so maybe if you're pulling out like a coding bot or something or coding relating related that might be helpful and so finally I'm just going to be showing you their text embeddings model I'm not going to be going too far into embeddings you can check out the other videos on my channel if you're new to all of this and new to embeddings and all of that I have some examples where I run through embeddings in depth with pine cone and open Ai and really break down what's going on and how to use them but this is just their version of embedding data so if you're going to be trying to make a application that has some sort of semantic search so say you have like PDFs or text that you want to search within this might be helpful for you and so similarly we can just go node three text embeddings and it will just take a moment and then what's returned if you're not familiar with it it will just look like a a bit of a mess potentially but um let me just make sure that I have this set up correctly let me just I'll use this Simba and Gekko hello world so essentially what we get risks a response of is it or it should be just an array of different uh encoded values so if we go ahead and look at our embeddings here you'll see all these so I think there's like 736 or something along those lines of values depending on the embeddings endpoint they are a little bit different I think open AIS is around 1500 where you'll get different values representing the piece of text so that's representing hello world within and embedding here so I'm not going to dive too much into embeddings but that's pretty much it for the video this is going to be on GitHub you can reach for this for some example code and as always if you have any questions or have trouble setting this up leave a question in the comments below but otherwise please like comment share and subscribe and until the next one