Codestral: Mistral AI's FIRST Coding Model

--- type: transcript date: 2024-05-29 youtube_id: zjdBuXQ-2BQ --- # Transcript: Codestral: Mistral AI's FIRST Coding Model in this video I'm going to be showing you code STW which is Mr ai's latest coding model I'm going to be pointing you to different resources on how you can use and try this out this model was designed specifically for code generation task as it mentions in the blog post it helps developers write and interact with code through a shared instruction and completion API endpoint now the interesting thing in the blog post is it's mentioned that it can be used to design Advanced AI applications for software developers Cod straw is fluent in 80 programming language including the most popular ones like python Java C C++ JavaScript it's nice to see that this was trained on a broad number of programming languages here it mentions that code STW saves developers time it can also complete coding functions write tests complete any partial code using a Fillin the-middle mechanism so you can imagine using something like GitHub co-pilot or continue or you can have something that will autocomplete your code as you're writing it to break down some of the technical specs it's a 22 billion parameter model so depending on your machine there's a chance that you will be able to run a model like this locally it is on that larger size generally speaking 7 billion parameter models work pretty well across most devices say in the past four or 5 years and obviously on newer devices it works quite a bit better assuming you have enough space and a reasonable amount of ram you'll be able to run this in terms of the context length it allows for a 32,000 token contact length that you can pass in and you can see on the human avow metric that this far exceeds even the Llama 370b model in terms of mbpp it does rank higher than llama 3 it doesn't quite reach deep seek coder 33b but it also is a considerably smaller model that's something to keep in mind in terms of some of these other metrics so the mbpp and then the krux ofal these are mostly python specific metrics for a model it size it is at the top of its category the one thing to know with some of the metrics are just a little bit shy of some of these other models is you have to keep in mind this is a 22 billion parameter model when you're comparing the SQL spider score this is comparing to a 70 billion parameter model and then is similar on the deepsea coder MPP python evaluation metric here it comes pretty close but this is also a model that's about a third of the size larger this model also has a significantly larger contact length that it accept so for programming task the more context the better the more that we can pass into a model and it has context of what we're trying to accomplish the one thing that I wanted to point out specifically on my channel so I'm primarily a typescript developer this model does outper form on the python metrics by a large margin across the board especially for its size but given that a lot of the content and AI applications that I buil are typescript related you can see that on the human ofal typescript metric the Llama 370b model is still at the top here this is obviously still a very impressive metric given the size of the model now in terms of the fill in the- Middle performance something like GitHub co-pilot or something like continue this is largely what those types of plugins use now with that being said with the typescript functionality this is pretty gigantic for JavaScript so the human fim so fill in the middle evaluation metric so this absolutely crushes it across the board so you can see for both python the leap from Deep seat coder 33 billion parameter this is a huge increase and then also even for the JavaScript model so to use this within JavaScript applications and within something like a tool potentially continue or Cod from source which I'll imagine they'll also be able to support this you'll be able to get some really good results all right so now I'm going to show you how to get started in a few different ways the easiest way to get started is you can just head on over to the chat on Mr AI to get started you can just go ahead and select code STW on chat. mistal do and then you can paste in the code so you can ask it to do particular things you can say improve the error handling in this let's just see what it gives here it's incredibly fast for response times and we can see it's starting to add some try catches to the original logic that I have here so I'm not going to test the model in depth in this video I'm just really going to show you different Avenues on how you can actually get started so if you want to gain access to their API just head on over to console. m.ai once you're logged into the console here you can go to code STW here and then you can request access to the model so right now you will have to request access I did have to put in my phone number and then shortly thereafter after I put in the confirmation code I was able to gain access they mentioned that this endpoint is going to be free for about 8 weeks and they're going to be gating it through this weight list to ensure the quality of this so if you want to gain access to the endpoints you can just grab the endpoint base URL right here and then similarly if you want to generate an API key you're able to do that just here as well H skated my API key here cuz that's one thing that I noticed in their console is they don't actually have that obus skated on screen here so now the one thing to note is let's say you are a GPU provider or together AI or something like grock or what have you you are able to still contact them for self- deployment if you're interested in doing that let's say you're company like together AI and that this is going to be a very popular model potentially and you want to deploy this you can go ahead and contact their sales as well so while it's nice to have that lchat interface where you can get started there are some really great options on actually including this within vs code if you want to get started they have an integration with continue dodev as well as tab 9 right now if you're going to be leveraging their API key it does have a rate limit of 30 requests per minute as well as 2,000 requests per day and it's going to be free until August 1st they have some good examples here on getting started Within their documentation but I'm going to show you how to get started within continue itself if I just head on over to a project you can search for continue within the extension Marketplace and then once you've downloaded it it's really simple to get started so you'll be able to see it within the bottom right corner here you can enable it if it's not enabled and then to actually get set up so on Mac the command is command option L and then you can open up this leftand pane here and this is going to be how you're able to interact with some of the different models that they have if I just go ahead and select that I want to use a new model you can select the mystal API and then from here you can just go ahead and copy your API key you can go ahead and paste it in right here I have an empty typescript file what I can do here is I can specify what I want to have generated I can say write me a Hello World Express server you can see very quickly there that we have our response back from the code STW endpoint and then what I can do here is I can just go ahead and select it so you can command shift enter on Mac to accept it and then o have the ability within continue where you can highlight a piece of code just like this and you have a couple options with command I you can say change this endpoint to hello next let's just put in a random word there you can see instead of that base URL now we have that mend Point pointing to hello next and then similar you can select it or reject it just like you saw there now the other nice thing with continue is say if I highlight a piece of code again you can also click command L you'll see that we now have this code block within this chat interface here so I can say something like what is this doing and then really quickly it's describing in really good detail exactly what that's doing so if you're familiar with GitHub pilot or more recently super Maven or any tools like that these code completion tools continue is a really great option cuz you can use code STW but you can also use a number of different models as well basically all of the different popular models you're able to incorporate into your editor and you're also even able to incorporate local models say if you have something running locally on AMA you can also incorporate those as well well that's it for this video I just wanted to do a really quick one on how you could get started with code STW but otherwise congratulations to the team I'm Mr AI on this new Incredible model that they put out for us I'm going to be trying it out over the coming weeks and months if you found this video useful please like comment share and subscribe otherwise until the next one