Chrome's Built-in AI in 4 Min

--- type: transcript date: 2024-06-26 youtube_id: v7mQ_eaT4Gw --- # Transcript: Chrome's Built-in AI in 4 Min this is absolutely amazing Chrome is going to be shipping AI models directly within the browser I'm going to be showing you how to set this up it is currently in an early preview program I'm going to be showing you how to sign up for the preview program and then how to set up the different flags and all of that once you have access the number of people that use Chrome is absolutely gigantic having developers that will now be able to access the API to set this up with just a couple lines of code is going to I think onard a lot of new AI experiences and not to mention this doesn't incur any cost at all since this is running completely locally on the machine of the client I'm going to run through exactly how to set this all up but the first thing that I want to show you is to get access to the built-in AI early preview program which I encourage you to check out from Google I'll put this link within the description of the video just fill out the form it took about 24 hours for me to gain access and once you gain access you'll get great documentation on how to get started with all of this I'm going to be showing you how to set this up with the development version but you can also set this up within the canary version as well go on over to google.com/chrome download it and go through the installation steps once it's all installed you will have to go through a few different steps which I'm going to be showing you the first thing that you're going to have to do and I'm going to put all of these different flags within the description of the video you'll be able to just copy and paste some to get started the first flag that you have to turn on is this optimization guide on device model you have to set this to bypass perf requirements and have it enabled and then the other flag that you'll have to set up which is right below it is going to be the prompt API for Gemini and Nano I'll also just show you the path to the flag if it isn't right below on your system just so you'll be able to access it it's again Flags prompt API for Gemini Nano just make sure those two things are installed once those are installed you can head over to wherever you have a console open and you can check whether it's available the way that you can check is if you go await window. doen create text session call that method if it comes back with the string of readily you will be able to start to interact now if you don't have readily that comes back what you will have to do is if you go like you see within the address bar that I have here it's Chrome colon components you will have to make sure you have the latest version for is the optimization guide on device model I want to show you how simple it is to interact with the API so to set this up all that you have to do is you can declare a variable and then you can await window. created excession there's also a couple methods here that you can access you can set up a session for the interaction that you're going to be having with the built-in Ai and then all that you have to do is await window. a create text session easy enough then from there if you want to stream out the responses you can access the prompt streaming method like I demonstrated in the outset of the video or if you just want to access the prompt itself you can just go ahead and select prompt if you just want all that text coming back at once one thing that I did want to mention is when you first call the model it might take a second for a response or depending on your machine but then for the subsequent responses if you're interacting with it say like in a chat context or something like that or it's being invoked an awful lot you will be able to get these pretty low response times so on my machine I was able to get as low as I think 43 milliseconds for the time to First token and then the whole response time is under a second to parse a relatively small ount of tokens I'm saying return in Json with the keys job name location and age I think this is going to unlock a ton of different use cases right you don't need to access any API keys or rely on any third party providers you're even able to access this offline like I already mentioned there's just a ton of really great benefits of having this right built into Chrome it's really straightforward to set up I'd encourage you play around with some of the different method types and then once it's all installed all that you need to interact with this is what on screen here let me know in the comments below the use case is whether you think this is interesting whatever you think put in the comments below I'm curious your thoughts on how you could potentially leverage this within your application or just your overall thoughts on something like this let me know but otherwise if you found this video useful please like comment share and subscribe otherwise until the next one