LLAMA 3: Set to Rival Top AI Models

--- type: transcript date: 2024-04-18 youtube_id: PDhNoLGaRBI --- # Transcript: LLAMA 3: Set to Rival Top AI Models llama 3 is finally here So Meta has just released two new models today an 8 billion parameter model as well as a 70 billion parameter model and both of these models are leading within their category but the one thing that is really exciting about the announcement today is not even just the models that they announced but the announcement about the model that they currently have training right now they have a 400 billion parameter model that is looking to potentially exceed both Opus and gbd4 in their current form they have a checkpoint within the training process for this new model and it's already showing an mmu that's just shy of both Opus and gp4 and mind you this is still training so it's going to be really interesting to see once this actually gets released and is open source what this will mean in terms of implications so just to get into it a little bit one of the key takeaways is now 8 billion parameter model is almost on par with the 70b parameter model that was released with llama 2 so it's not quite at 70b but for the scale it is awfully close it is within the range within Striking Distance of something like llama 70b so very exciting llama 3 exceeds both Gemini and clae 3 Sonet across a number of different benchmarks and the thing that's impressive with this is a lot of people find that clae 3 Sonet really is on the scale of GPT 4 so one thing to know with these models is they're not mixture of expert models so in terms of the inference speed at least on the higher end for like the 70b parameter model it's not going to be as fast as something like a mix model or something like that in terms of some of the details these models are trained on 15 trillion tokens of context which is just astronomical the knowledge cut off for the smaller model is March 2023 and then the larger model is December 2023 now one thing I have to point out off the bat is both of these models only have 8,000 tokens of context that you can pass in which really isn't a lot in today's climate when you have Google Gemini going as high as a million and you have Claude I think going as high as 200 ,000 GPT 4 128k 8,000 really isn't that much especially in applications where you might have a longer system message and this is a big constraint considering if you're trying to build a reg application with a system message you're really sort of bound to the smaller window at least for now now that being said I wouldn't be surprised if we see that context length being extended with new variations of the model from the open source community over the coming weeks I did see this on Hacker News which is an interesting comparison when you do compare these models to something like gbd4 obviously they're not quite there they're really Best in Class in terms of om Source models but the thing that's really interesting is this 400 billion parameter model that's coming out we're going to have this 400 billion parameter model that is going to meet or exceed gbd4 so they have the checkpoint that they mentioned like I already talked about but we see on the MML U score it's already at 86 on the instruct model when you compare that to GPD 4 it is 86.4 by the time this is done training I would very well expect that this would exceed GPD 4 so what are the implications of this this is going to be really interesting to see we're going to have a number of different providers being able to provide these open source models from the hyperscalers what I'm really excited about is to see all of these different models on grock so Gro if you're not familiar they have the fastest lpus and I'm going to be really interested to see how fast they can make the inference across all of these different models if you want to try this out you have a couple different options with a ton more legely coming out you can head on over to llama 3. replicate dodev they have both the 70 billion parameter as well as the 8 billion parameter model that you can try out so I'll just demonstrate it here if you want to try out the 70 billion parameter model I'll just show you what it looks like here so you have this sort of chat gbt like interface here that you can play around with and then similarly if you want to run the 8 billion parameter model you can just go ahead and run it just like that so another option is together AI where you can go ahead log in make an account and you'll be able to play around with it within their playground so another thing to note with the model they are using the openai tokenizer for this there is a larger vocabulary and they've also trained it on more during pre-training so in terms of some of the other pieces here it will be available on AWS Google Cloud hugging phase and a ton of other platforms Azure snowflake Etc now the one thing that's interesting I went looking for their research paper and they mentioned that it isn't going to actually be coming out until they're done training llama 3 which is that big gp4 level model I'd encourage you check out some of the Twitter threads out there there's a ton of information out there of people just diving into this right now another impressive chart that they highlight within the blog post is how how it Stacks up to a number of different popular models if we compare it to Sonet mistol or GPT 3.5 we see that llama 37b wins across the board across these models it's pretty exciting where they're at already it's even more exciting where we're going to soon be with this new open source 400 billion parameter model but I think for a lot of people and a lot of use cases this new llama 370b variant as well as the 8 billion parameter model are going to be incredibly valuable which pretty EX that now we have these open source models that are really catching up to these closed Source proprietary models it was less than a couple months ago that we had CLA 3 come out and now we already have these open source models that are exceeding their middle tier model which is pretty exciting exciting stuff to come from meta for giving this to all of us everyone within the open source Community as well as all of the different developers I'm excited to see what this will look like over the coming weeks and months and last I just wanted to show you that ama now has support for the Llama 3 Model so if you head over to llama.com download it if you haven't already go over to models click llama 3 and then you'll be able to just copy that command here if it's the first time that you're running it paste it within your terminal and it will install you can just run it so if I say hello world write me a story you'll see that it just takes a moment to boot up and then it will start to respond for you so there you go you have llama running locally so hopefully you found this video useful if you did please like comment share and subscribe and otherwise until the next one