NVIDIA's Llama Nemotron Nano 8B Vision Language Model

--- type: transcript date: 2025-06-25 youtube_id: YarMz4vl6qg --- # Transcript: NVIDIA's Llama Nemotron Nano 8B Vision Language Model In this video, I'm going to be taking a look at Nvidia's Llama Neotron Nano Vision language models. In terms of benchmarks on the OCR Bench V2, which is basically a big test that checks how well the AI can read and understand text from all different sorts of images, whether it's signs, receipts, diagrams, charts, basically all of those different component pieces. Right off the bat, this is an open source model. NVIDIA's state-of-the-art VLM uses radio vision encoder and Llama 3.1 as the backbone for the LLM. And if we look basically across the board in terms of the aggregate score across all of these different metrics as well as even close source models like Gemini or GPT4, we can see that this model ranks number one. And the really impressive thing with this is just an 8 billion parameter model. Whereas on this same benchmark, we have models that are considerably bigger. for intern v2. That's a 14 billion parameter model as well as intern VL 2.526B. That's basically a model that's three times its size. Not to mention these closed source models where we don't necessarily know the size of these models. But both Gemini as well as GP4V are way bigger than these models as well. In terms of text recognition, we can see that this is just shy of Quen VL7B. In terms of text referring, we can see that this is considerably higher than all of the other models at 69.1, whereas we have the second best model at 39.5. We have text spotting again at a huge leap from 61.8 over Quen 2's VL7B. While this model doesn't outperform on every single metric, just to see some of the size of the leaps of some of these different metrics, it is incredibly impressive. And the one thing that I do want to note with this is in terms of some other calculations like mathematical calculation. You will notice that Gemini Pro is better at math. The thing with Gemini Pro is this is a massive model. So, it's going to be running on a ton of GPUs. It's also closed source and going to be considerably more expensive to run. Overall, this is a really great option in terms of those document processing use cases because, as you might know, running inference is probably one of the more costly parts of your infrastructure, especially if you're doing this type of processing at scale. So, being able to have a smaller model that's more efficient unlocks a ton of different use cases. Next up, in terms of accessing the model, you can download the model on hugging face right now. Alternatively, if you want to try it out, you can go to build.envidia.com nvidia.com and you can try it out on their serverless GPU platform. And the nice thing with this is you can use this free for development. In terms of the playground itself, it's super straightforward in terms of how to get started. You can go and grab your API key. You will be able to get this for free for development purposes. And then you have the option where you can test all of the different models directly within the interface here. Or alternatively, you can grab the relevant script and plug this directly within your application. And the nice thing with how the platform is designed is it is set up in a way where it will be supported with the OpenAI SDK. You can just go ahead and swap out your API key, the base URL, as well as the model string, and that's all that you need to do to change it out from something like one of the GP40 or whatever it might be. So now, in terms of actually demonstrating the model, there are a number of different examples. I'll show you a couple of these just to show you how the model works. Within this image, we have a number of things going on. We have two different graphs. We have a ton of different text. We also have the y-axis of some of these different numbers. Then in terms of the question, it's a non-trivial question as well. For the mixture of expert switch XXL training, what is the speed of H100 over A100 and H100 with NVLink over A100? We can see that the H100 over the A100 is 5x the speed with the H100 with NVLink over the A100 is 9x. So, if we take a look at the image here, we see exactly what it's describing. Now, I'm going to test this on some data that I go looking for. I'm going to look on the SEC filings page for Apple, where there is often a ton of really dense information within these Edgar filings. Just to give you an idea, if I take a snapshot of the financial statement of Apple here within the screenshot, what I'll do here is I'll come up with a question that is going to be specific to one of these rows. I'm going to upload the image and what I'm going to say is what is the year-over-year delta for net sales in products. Within here, we have it describing what the year-over-year delta is. And then from there, we have the equation that it came up with. And then here, we can see the year-over-year delta for net sales in products is 2.74%. Now if I take a look at the financial statement in terms of the equation we had 68714 minus 66886 and then divided by this number so on and so forth. The fact that it can not only just get the information accurately from the image but also be able to respond and reason in a coherent way in terms of actually how to break down that problem. It's super impressive especially once again given the size of the model. In terms of some of the other aspects of the model, it actually does support a number of different input formats. So, you can input images like I just demonstrated, but you can even add in videos. That is one thing that I don't see within the playground quite yet, but if you are going to be running this yourself, it would be really interesting to see the results on how it performs with video as well. One thing to note with the model in terms of the context window, it does have 16,000 tokens of context. That's just one thing to be mindful of. In terms of getting started with the model, they do also have a quick start guide which you can go through that is on hugging face here. You can easily install all of the dependencies and they do have a great example here with a number of different images as input being passed in. Basically, in other words, the quick start has exactly what you need in terms of getting started. So, what's really interesting here is they lay out how they use different internal data or public data for different parts of how they train the model. One way they leveraged synthetic data sets was for specific tasks like tabular data understanding. That is potentially a use case where synthetic data sets could be useful, which is pretty interesting in of itself. Now, another handful of use cases in terms of how you could leverage a model like this. Here's an example of asking the model to extract the table in the image as HTML. Where HTML can be helpful is for things like rendering within a chatbot or potentially asking it in the form of markdown or what have you. Here we see it's streaming in this really nice HTML table of all of the different things within this technical specifications table. And finally, just one quick demonstration. Say if you sent in a newspaper or a report, the one thing that all of these different document types have in common is they're unpredictable. They're going to come in different formats. For a model to actually be good at this is it really has to generalize across a wide array of different tasks. And within here we can see a bulleted list of all of the different technological breakthroughs of the NVIDIA hopper. We can see the H100 tensor core. We can see the transformers. If we just go through the list here, we can basically see we have the perfectly extracted information of all of these different headings from this image. Now, in terms of ways that you can leverage this model, you probably have a ton of different ideas, but just to list out a number of different use cases. This could be for processing invoices or receipts, contract and legal documents, healthcare and insurance automation. So, and one thing to know with these types of models is there is an absolute ton of different applications that you could potentially build with this type of model. I saw recently someone had a simple bank statement conversion app that was making tens of thousands of dollars. And this is the type of model where you could potentially make something like that if you are interested in building out an application for yourself. Kudos to the team at Nvidia for this release. And if you found this video useful, please comment, share, and subscribe. Otherwise, until the next