ChatGPT: Alpha Plugins (Web Browsing with GPT-4 & Code Interpreter)

--- type: transcript date: 2023-05-04 youtube_id: YVgzLCLwyYc --- # Transcript: ChatGPT: Alpha Plugins (Web Browsing with GPT-4 & Code Interpreter) all right in this video I'm going to be demonstrating some of the alpha plugins for chat GPT so a bit of context for individuals that are curious because I'm sure many are how do you get access to this so the way I got access is I made a request about a month ago when gpd4 came out I requested access to the gpd4 API I requested access to use the plugins I requested access to develop plugins being a developer and so I sent several form submissions and the lead time I don't know if they're just rolling it out through the list bit by bit or if it's going in batches right now but I got access on the weekend I believe at some point when I noticed that these were available so I'm just going to cover a handful of quick examples I'm going to go through a series of tests for the web browsing I'm going to go through what I consider an easy easy task and then ones that are maybe a bit more intermediate and then Advanced potentially in terms of how it it could potentially handle it so the first thing that I just wanted to demonstrate is asking it a simple question that would need to gather information from today so what do I mean by that so I'm just simply asking what the what is the weather so if I say what is the weather so because llms they're trained on data sets that have a lag that's the really interesting part with uh tying in these llms with the ability to browse the internet is all of a sudden instead of having data that was trained in 2021 you can have that Corpus of data but then also reference pieces from the internet if your query requires it so as you see here I've noticed this a fair bit is even if you ask something it uh pretty seemingly specific you'll see that it's asking questions quite a few questions and checking on various web pages and often it's checking the same web pages um and uh you'll see it's just sort of hopping back and forth through different and sometimes the same sometimes not available websites and then ultimately it somehow landed in San Antonio Texas so I'm not based out of San Antonio uh maybe the server is I'm not sure if anyone's uh dived into that aspect of why it would assume um that as the location but let's just say I'm getting a bit more specific so if I say what is the weather uh near where I'm located in Toronto is the weather rather in Toronto and I'll just ask it here so there are some nice things so as you saw um when you expand this tab here you can open the links as they go through but sometimes there are errors it would be nice if I could click something here to see what that error is and then sometimes there are some very seemingly redundant like going back to last page commands like sometimes I've noticed it will just say scrolling scrolling and it will fill the screen with scrolling on a page that really shouldn't require um you know seconds and like 30 seconds of scrolling for instance so again um now if I just go back okay so here is the weather and that's accurate so that's a more succinct what I'd Sort of hope that this final product or version of this product would be released to the The Wider wider public would give as an answer so now another question and use case that I thought about is what I've noticed with using chat GPT and just the API for the the models is sometimes it doesn't give me correct information in a coding context so an example of that is when there are breaking changes within libraries or dependencies of libraries sometimes it will tell you to do something in a particular way and it doesn't know the version that you're working on if it's newer than what the llm was trained on so I've made a handful of videos on Lang chain recently so just to demonstrate it with Lang chain and in this one I'll I'll show you with the gpd4 example I'll say what are the newest features released in Lang chain so Lang chain if you're not familiar as we wait for this to load so it's a new framework that you can use on the back end to tie your llm into a whole host of services and build your own Solutions on how you want to interact with these llms whether it's browse the internet or have a conversation with your files I've done a handful of videos on this I'll put a couple links in the description and maybe pin them on the the video here that you can take a look at if you're curious but this was something that I did earlier just to get a sense on okay a little bit more advanced maybe not Advanced of a prompt still pretty simple what I'm asking but it's uh a bit more involved than just saying here's the weather here's a couple details about the weather and so while that's loading on my right side I'm just going to show one thing on my left here that I I do think is a nice feature but not something that I've noticed as consistent at least not yet and that's that it actually has this annotation for the link so there are references when it gives you information back from crawling the web so instead of worrying whether there's a hallucination there from the llm it will give you that reference so I think this is pretty interesting because with these models knowing that hallucinations are a problem being able to have a reference for maybe material that an llm is you know sort of quote less certain about it could try and make an attempt to validate that with information on the internet but I mean maybe maybe not right because you know information on the Internet is is not always true itself but at least having that reference will um will or could help I should say so the other thing um here and this is a good demonstration of this is sometimes it just doesn't finish it looks like it's loading so the loading spinner is there but if you notice that these refresh buttons as soon as that stop icon goes away this is just done um so I'm gonna have to ask this again so that's one thing with the Alpha version it's not as polished as Chachi BT it does give you that feel very much of like this is like an alpha version like it's like you're like an internal company user of this product where it's you sort of get a sense on where it's going you can give feedback like I'm doing right now but you sort of know what you're in for like this isn't uh you know like I'd expect a Polish polished version of this if it was it'd be released to everyone right now so if I just go ahead and save and submit let's just see give it another opportunity here so Lang chain new features and so a different result so that's another thing too uh similar to llms um it's not going to consistently give you or perform the same question the same way so this is actually what I would hope uh or would have hoped in the first example that it would have done is actually go straight to either the documentation or their website to try and get information on the releases because that is probably better than just trying to Google or Bing I'm not quite sure what they're using on the back end to search you know I might assume that it'd probably be Bing with their ties to Microsoft and all but uh yeah let's just see what what happens here so you'll see okay it will go in and out of pages it sort of works like Auto GPT or baby AGI except it's more likely to give you a response um if you've used baby AGI or Auto GPT you'll know that sometimes it will just keep running and running and and just create like these recursive Loops of doing all these tasks you might ask at the weather and it will start giving you like the you know history on the Eiffel Tower or what have you so very very early for some of these tools but again similar to Auto GPT or baby AGI you sort of know what you're getting into these are tools that are literally weak sold in the case of this feature this web browsing feature I bet you with even internally with an open AI it's within the past number of months certainly not years they've been working on this so we'll keep letting this run here on this tab but while this is going actually we see this one also stopped so we can sort of see I like to look through the pages and sort of get a sense on like what is it doing like is it continually going to the same page like that for instance I feel like you know that could be a condition potentially programmed in I'm not sure what they're using to browse the web like I noticed I see like you see that they're clicking on things they're scrolling things so I'd imagine maybe something like selenium or Puppeteer but it would be curious to know how they're doing this and maybe just setting in some base conditions like don't visit the same page like store these things in a vector um you know uh just save them in memory for this session while the user is going through it and don't continually go to the same Pages again and again because you see here here you know it hit the same page at least five times here so um like I mentioned so I'm just going to go in and refresh this tab here I'm going to go in and close some tabs as well and what I'm going to be doing here is just quickly showing you the code interpreter so you've probably noticed or maybe you have maybe haven't noticed some examples of this already now the interesting thing with this is what what some users or at least what I didn't see at first is that you can actually upload files so you can upload something like a CSV now this code interpreter is using python I did ask on Twitter and did not get a response on whether they'd consider putting in a V8 engine where it could please node developers JavaScript developers like myself to be able to generate node.js code and interpret it on the the fly would be pretty amazing but nonetheless python is obviously the larger so Python's obviously the larger group within machine learning at least in terms of uh the programming language that's used so it's completely understandable why they went this route but so one thing with this you can upload files and it's not the python file per se that you're uploading is you could upload a CSV of data for instance so if I just go ahead and upload so I have some of my YouTube data here so none of this data is a secret by any means I started on YouTube a handful of months ago I'm still a relatively small Creator as you see so if I just upload some of this data I didn't get a comprehensive data set I could certainly make this data set quite a bit bigger in terms of what I exported from the YouTube analytics page but just to give you a bit of a reference so if I go ahead and upload this file now one thing to note with this which is pretty cool is you can upload files that have a huge context so I've seen examples of people use you know like a 20 some megabyte file whereas if you tried to paste in a 20 megabyte file into the context window or send that to the API for gpt4 or or what have you that's going to be way more tokens than can be what is handled so this is interesting where it's like say you have a large set of data and you really wish you could just throw it within Chachi BT and get some answers this sort of will take a look at the data look at the structure and then think of Solutions on how to parse it so do you want to make a chart do you want to create like a correlation coefficient or whatever you want to do so okay so you'll see once I uploaded here you'll see okay so it says let's get started by loading the CSV data and start taking a look at its contents I'll use the pandas libraries to do this so the pandas is what like a data scientist would use uh for instance so you see okay this CSV contains data about various videos information title publish time Etc uh let's get a better understanding of the data how many rows Etc so it's starting to sort of infer you know how it's going to or how you might want to use it and also how it's thinking on how it's going to use it and then from there once the data is loaded in there you can do things like create a novel charts and provide any insights you might have from this data so you can see like by creating charts like on the Fly and having a python interpreter like you can really have limited or no coding experience and be able to come up with solutions that you know could have otherwise taken a programmer or a data scientist to develop so and the nice thing with it is it will show you its work so you know if you're new to programming you can certainly use a tool like child GPT or GPT based model to help you with learning and the nice thing with this is it's showing you its work is you can always take this code and run it locally if you want as well so if it really does like a bang-up job of what you're trying to do you can save that down store it um and uh save it for later so you see this is sort of working in the background and here we still don't have an answer for our simple Lang chain question so like I said you know I just wanted to give sort of a raw example so if you are really eager to use this I just be patient you know don't worry too much about it it's still very very early you might not find it as useful in its current state as you might think it is in a lot of these polished Twitter examples you see I'm sure a lot of those polished examples are running into this you know the same friction that I am where things just don't work or you have to you know spend a lot more time than simply doing a Google search for something so if I just head back over here um you'll see okay so it started to um go through um creating what I asked for so some novel insights on my data so a bar chart showing the top 10 videos with the highest average percentage viewed I assume what it was saying so a scatter plot showing the relationship between the number of views so the histogram okay so let's just sort of see what it has here so we'll see that here we have the top 10 views with the highest view percentage so the interesting part with these is so a lot of these with the really high view percentages now these are YouTube shorts which I did when I initially started my channel but I haven't done for quite some time so the thing with shorts is they can have an average view percentage over a hundred so if you're not familiar with YouTube and how it works on the back end you might see this and say well this data can't be right like there's things over 200 percent here but the reason why this is um important is because the way that this works or maybe not important but why I should point this out is once a user gets to a hundred percent they might have rewatched it once so on average this video is viewed twice and this is you know a video down below here it might be viewed all the way through and then some on average so then another interesting Insight okay so the relationship between the views and impression uh click-through rate so this is something that I've heard like huge and like the biggest YouTubers will do is really dive into the data analytics um like a Creator like Mr Beast is sort of notorious for uh creating insights based on his data and really trying to refine everything like you know what's the what's a good length for a title or What's um you know the the color tones you should use on a thumbnail Etc and so on so now going through this it sort of gives you a really quick ability as a Creator to leverage um you know what you might have like I mentioned otherwise needed a data scientist for so and then also okay let's see what else it came up with so distribution of watch time hours for all videos so yeah so this is um my watch time um there just trying to see what's going on here so the frequency I'm actually not sure what's going on here but if it is the watch time of my video so I have one video that's like the vast majority um of my watch time hours so it's my GitHub in seven GitHub co-pilot rather in seven minutes and it is something like 1500 watch hours so that's my one video that really just took off um before uh for my channel so far so you can say you know you get a bit of a rundown on what it's doing here but I could just go back here and I only provided it like a basic example of a CSV there's not many columns in that CSV you know I think there's less than 10 columns in the one I uploaded but you could add all sorts of data analytics that YouTube provides and really dive into maybe how you could optimize videos if you wanted so um maybe I'll give gpd4 one last chance and then um I'll just quickly run through while I'm doing that some of the new plugins that were just released I believe yesterday um some of the third party plugins I should say so let's think of one more example so um uh research um some of the trending libraries on GitHub right now so I did do a prompt like this earlier where I asked specifically what libraries are trending within Ai and it really had a hard time with it so similar to that Lang chain example that was actually the example where it was like scrolling scrolling scrolling you know it just went on and then stopped I tried it a couple times it didn't work but something like this I would expect it to be able to figure this out so um it will be interesting to see when they start to roll this out or whether this is a focus internally for them on like refining sort of how it interacts with uh selenium or Puppeteer or whatever they're using to query the internet maybe they're using a um some some other library or something they built themselves to to do so but there really seems to be or is it you know in my opinion there is a fair bit of work there to really refine this and and polish it up for the broader release but pretty exciting some of the examples when I have got it uh to to work it can be pretty helpful but I think the hit rate it's like probably under I don't want to be too harsh but I also want to be honest is probably under 15 right now like it there's a lot of just like frustrating uh instances um like this where you're saying you know either there's too many requests or it just stops and it doesn't really give you the um Insight that it stopped but yeah so I'm going to leave the gpt4 for now and I'm just going to hop over to the plugins so plugins is also in Alpha there are some third-party plugins that they have um released there's more and more coming I'd imagine this will just start to fill up so Zillow is new Tasty Recipes now these are always like to each their own there some are going to be more relevant to others or some than others rather I thought it was interesting even Lowe's like a company like Lowe's saw the value in putting their plug-in within the ecosystem so I'm going to wrap it here if you have any questions or if you have any examples that you want me to do I might do a follow-up video in a number of weeks seeing the state of this how it's progressed how it's improved maybe give you more hopefully more examples where it is a bit more impressive with the browsing but yeah let me know your thoughts in the comments things you want to see prompts you'd like to experiment with if you don't have access to it yet I'll happily throw in some prompts that that you're you're interested in seeing and include them in in a future video so hopefully hopefully you like this if you did please like comment share and subscribe and until the next one