OpenAI’s Operator Hitting Walls? Here’s the Key to Full Web Access

--- type: transcript date: 2025-02-16 youtube_id: W1QydKNqU4E --- # Transcript: OpenAI’s Operator Hitting Walls? Here’s the Key to Full Web Access I came across this post that got quite a bit of attention on both Reddit as well as X which is pretty interesting this is chat gpt's new operator feature and what the video demonstrates is of it landing on a page that requires the user to solve the capture that we've all had to do before selecting different crosswalks bicycles and we do see that operators struggling with this after seeing this and I wanted to test on operator for myself on what different websites were actually blocked and one of the things that surprised me is how many sites were blocked so here I have operator all set up and in the first quer I have I want to find the latest blog post from open AI here we see that it went to Bing and as soon as it tried to navigate to the open AI Newsroom we are blocked by cloudflare in order to actually get through with operator I had to intervene to be able to get this information even using chat gbt's own tool that's one thing to note with some of these AI search engines is they're definitely not a silver bullet trying to access information if I say what is the latest from Reddit this is another example of one of the most popular sites on the internet where you're just not going to be able to access it and this isn't just isolated to chat GPT operator there are a ton of these different search sites out there where it's going to block things from LinkedIn to Google search this is just a demonstrate of some of the limitations of the current state of AI agents and even if I try and access Amazon just the plain amazon.com website it promise me with the capture even if I specify to have the agent and say you do it you solve this capture it will say I can't complete captas but if you could handle it I'll proceed with searching for a rain jacket on Amazon with that being said if you are interested in building out an application that you do want to have access to a wide variety of data on the internet I'm going to show you one solution for that in this video I'm going to show you bright data scraping browser what you can do is you can make a account on bright data and within here I'm going to instantiate a new scraping browser within this scraping browser I'm also going to enable the capture solver I'll go ahead and I'll add this here once you're set up you're going to have this proxy that you can access and this is going to be what you use whether you're using Puppeteer playright or selenium to access those different websites once we've instantiated the browser this is what the interface looks like now you can use this all programmatically so you can just go and access the code stamp it for not Jazz python C whatever it might be you'll be able to use this in whatever environment or application that you're building just to demonstrate this here if I go and I try and access Reddit just as the first example here and I run this now what's happening under the hood is their browser is mimicking human behavior there's Mouse movements scrolling typing and which in essence mimic the behavior of real users as we can see we got this response from Reddit we can see the news from an hour ago now if I go back to operator that was one of the sites that we weren't able to access another one that we tried was the opening ey news site which funny enough even within their own tool was a Blog if I put in the URL to the open AI Newsroom we see it's running it's connecting to the browser it's navigating to the site again there was no issue in being able to access this data one thing to not with this when we were using operator we were blocked by cloudflare to access this page but when we tried Reddit it had a completely different mechanism where it blocked us from accessing the page here's just yet another examples we were blocked with the capture from Amazon we saw here it was able to again successfully get the response the other thing to note with the scraping browser is this is going to wait for all of the JavaScript to be invoked and rendered on the page before it takes the action to return the resols to you and where this is important is because the web is increasingly JavaScript heavy with all of these modern Frameworks effectively having to load the JavaScript before websites function as we would typically see them finally if I try one more example of trying to access LinkedIn I'm able to access that with the scraping browser just a run through on how to set this up you can install pupper core and with pupper core you don't actually need to install that chromium instance locally since we're going to be accessing it through for this proxy URL and what you can do with this is specify the proxy that we're going to be using navigate to a new page and if you're not familiar with Puppeteer or selenium or any of these tools they're effectively a synthetic browser that you can control with code here we can see an example of Us opening up that page now if there is a capure on the page to be able to solve that capture all that you need to do is put in this little bit of code where we can client send this capture. solve detect the timeout and then it will return the status back to us where this can be useful say you want to go to Amazon but you don't just want to go to a specific page what you can do with Puppeteer or these synthetic browsers is you can programmatically determine how you want to navigate the site similar to something like how operator works just to sum up the problem set you could set up your application in a way where every time it encounters a capture it will send some humanin thee Loop feedback and instruct you to accept and fill out that capture for you but ideally with all these agend applications out there we would just have a tool that works and doesn't need us to interject every time we need to solve a Capa or click a check box if you're building these types of applications and you are getting blocked by captas or having IPS being banned this is a solution where you don't need to worry about the nitty-gritty of all of those different aspects the other nice thing with this is since it is built with Puppeteer selenium and playright you don't need to incorporate a new library or rewrite a bunch of your logic if you are using one of these libraries already all that you have to do is effectively swap out your url to connect to the scraping browser that you've instantiated and you'll be able to navigate the site you can even take screenshots if you want you can even build out your own version of something like open AI operator but being able to access some of these websites that open AI doesn't allow you to access within their interface otherwise that's pretty much it for this video I just wanted to show you an option on how you can access a number of these different sites if you're running to issues like captas or being blocked with IPS or what have you bra data is a really simp Simple Solution whether you're building an AI application or if you're doing some form of data collection for your application this is a reliable way on how you can access these different websites otherwise that's pretty much it for this video if you found this video useful please like comment share and subscribe otherwise until the next one