---
type: transcript
date: 2025-10-28
youtube_id: skut607JoOA
---
# Transcript: NVIDIA's NEW Open Source Nemotron Nano 2 VL Model in 5 Minutes
Nvidia has just dropped a new AI model that can watch videos, read documents, and reason through visual problems, all while being four times more efficient than existing models. And it's completely open- source. NVIDIA has just released their latest in the Neotron series of models. Nano2VL is what they're calling this model. And this isn't just another vision model. This is a 12 billion parameter model. It can read and reason through complex documents. Think things like invoices, contracts, or medical records. It has the ability to analyze multiple images all at once with a visual Q&A. You have the ability to understand and caption long videos and you have the ability to turn on and off reasoning depending on your needs. And what makes the model special are a number of different things in terms of the architecture. One of the techniques that they use is called efficient video sampling that reduces tokens by 4x. Under the hood, it uses something called hybrid transformer mamba architecture. Now, I covered this on a previous video, but just to touch on this again because it can sound a little bit like gibberish in terms of what it means. But effectively what this means is what's really great with transformers is they have a great understanding at the context, but they're slow with long sequences. And part of the problem with that is, especially as you start to add in tokens that include video, this can often fill up that context window quite quickly. And that's where the mama architecture really shines cuz it's lightning fast, but sometimes it can miss nuance. And by combining these together, you effectively get the best of both worlds. Just to touch on the Neotron family of models. So if you haven't heard of Neotron models before, this is Nvidia series of openway models. One of the nice things with Nvidia models is they don't just provide the weights of the model, they also provide quite a bit of information in terms of how they're actually trained. And they provide a tremendous amount of research and make it readily available with a permissive license that you can use. They have everything from nano models all the way through to ultra models. things like 235 billion parameter models and they also have smaller models that you can run on something like consumer hardware. Now, in terms of what actually comprises a Neotron model, it varies depending on the model a little bit, but this gives you a general overview in terms of how they're actually built. The number of tokens, the number of samples, the compute hours, the number of research papers that are available across all of these different models. It is a very rich ecosystem that they've developed with these Neotron models. Everyone knows Nvidia's hardware story, but what's interesting with Neotron and now actually building these models is they can actually optimize these models for the stack of the different chips that they have. And the benefit of this is we see this with companies like Apple where they have both the hardware and then they have the software. And the benefits of that is they're able to create very novel techniques and architectures that work quite well with one another. Now, obviously they make chips that are just generally available and work across all of these different major labs, but being able to be on the inside, so to speak, is definitely a benefit. Being able to have the hardware teams as well as the AI researchers that create these models all working together effectively, it gives a really good sense in terms of the capabilities and the directions in terms of these models and where they might go. Now, just to touch on some of the other aspects of the model, this model is best-in-class for OCR as well as chart reasoning. And when we look across the board for the different benchmarks when compared to the previous version, the open source Neotron Nanov model, we can see the model outperforms on basically every benchmark that they have stated here. But what's really interesting with this is with the hybrid transformer Mamba architecture is the big piece of this story is while the performance is increased from the previous model is it is way faster. All in all, it's just a more efficient model. So you get much faster speed while you don't actually have any degragation in terms of the performance. Now in terms of some of the use cases, so I mentioned some of them already. Say for instance, if you want to gather insights from particular documents, be able to summarize different documents, you can leverage it from that. If you have any tasks that involve multi-image reasoning, you can leverage this model for that as well. And then finally, what you can do with this is dense video captioning. Last, I want to demonstrate a little application that I'll open source just to show you the capabilities of the model. The way that this works is you can download various YouTube videos and then once you have them downloaded, you can pass them in as a part of the payload to the Neotron model. Okay, so just to demonstrate this here, if I grab a video from my YouTube channel and I paste it within here, I'll click to download the video. It will just take a couple moments to download the video. Now, the one thing that I do want to note is you will be constrained by the token limit of the model. Just be mindful of that when you're actually trying this out that you don't pull down a super large video or anything like that. So what I can do with this is I can say I want to have a summary of the video. What's happening when I send in this query is not only am I sending in those handful of words, I'm actually sending in that whole 5-minute video. And the model is able to have all of that context both visually as well as what was spoken within the video. Here is the summary of five bullet points. And then last but not least, I'll ask a question of how could I've improved the intro of this video just to show you one demonstration in terms of how you can actually leverage video data for different applications. And then in the response I have you could have included a brief overview of Chachi PD's features and capabilities to provide viewers with a quick understanding of what the video will cover. That's just a handful of ways in terms of how you can leverage this. I'll also put this link on GitHub if you are interested in pulling down the NextJS application. Kudos to the team at Nvidia for the latest release and for open sourcing so many of these great models. But otherwise, if you found this video useful, please like, comment, share, and subscribe. Otherwise, until the next one.
