I didn’t realize AI could interact with browsers like this already (guess I’m naive). Isn’t this setting up for the scenario where the AI is duped into logging into your bank account and transferring your money away? Not sure I have enough trust to allow an AI to touch a browser.
People are already going full Leroy Jenkins with this stuff, and OpenAI, other labs are snarfing up their usage data. Hopefully with their brave sacrifice, they can figure out all the security pitfalls before it becomes common enough that someone with a clever jailbreak ends up pulling of a billion dollar heist, or orders pizza for half the country.
It's 100% absolutely not safe yet. You can effectively copy and paste Pliny prompts and pwn any of the frontier lab models. Anyone with a little time and creativity can tailor a unique one and set hidden text traps for AI browsers or agents, and depending on what access you've given the software it could be very dangerous.
There are folks on X running vibe-coded Polymarket arbitrage bots playing with hundreds of thousands of dollars. Some people have pretty wild risk tolerances!
That's a valid concern. I took a more constrained approach for web searches for exactly this reason. Instead of giving the LLM full browser control, I built a Firefox extension that only handles web search client-side.
When my local LLM (llama.cpp) needs to search, it opens DuckDuckGo in a new window, loads the result pages in tabs, extracts content with Readability.js, and feeds it back. You stay in the loop - can see what's loading, solve captchas if needed. Less autonomous than Comet/Playwright, with a narrower use-case, but also less risk.
It's totally setting up for exactly that scenario. You need to ensure the browser that it uses is totally unprivileged if you're going to do this, or at the very least that it can only access a small set of trusted destinations.
I tried it. My Perplexity premium expired, maybe that is it, but it barely did anything.
When I put prompt you suggested, it did open Perplexity in Comet and then I guess didn't get response even though Perplexity did research, so it used regular search mcp to get results...
It is cool idea, this is what I would like to have, something to automate boring stuff. Find all LinkedIn connections that are not active and remove them from my network for example.
I don't think it is your mcp or code, as tech is just not there yet. It is much easier to accomplish this through other automations.
I was going to ask what makes this better than just using Playwright and this largely answers that question. I will have to try it out and see how it compares.
I haven't really had luck with MCP in general for quite a while though. I have just been using Google Antigravity for most of my vibe coding needs.
I've used chrome devtools mcp successfully to do all kinds of advanced in browser tasks, agents like claude code can write js and inject it into the context in a live browser and do all kinds of neat tricks. I've used this extensively in gemini-cli.
It's 100% absolutely not safe yet. You can effectively copy and paste Pliny prompts and pwn any of the frontier lab models. Anyone with a little time and creativity can tailor a unique one and set hidden text traps for AI browsers or agents, and depending on what access you've given the software it could be very dangerous.
When my local LLM (llama.cpp) needs to search, it opens DuckDuckGo in a new window, loads the result pages in tabs, extracts content with Readability.js, and feeds it back. You stay in the loop - can see what's loading, solve captchas if needed. Less autonomous than Comet/Playwright, with a narrower use-case, but also less risk.
Its still a prototype though: https://github.com/tbocek/llm-local-web-search
When I put prompt you suggested, it did open Perplexity in Comet and then I guess didn't get response even though Perplexity did research, so it used regular search mcp to get results...
It is cool idea, this is what I would like to have, something to automate boring stuff. Find all LinkedIn connections that are not active and remove them from my network for example.
I don't think it is your mcp or code, as tech is just not there yet. It is much easier to accomplish this through other automations.
I haven't really had luck with MCP in general for quite a while though. I have just been using Google Antigravity for most of my vibe coding needs.
https://chromewebstore.google.com/detail/blueprint-mcp-for-c...
Doesn't lost plugins anymore. I'm sure I installed playwright using that menu, but now it lists no plugins (and the plugin can't be found locally)
However, claude add mcp and /mcp still works.
I trust Claude in Chrome a lot more, and I trust my own hands and eyes most.
What's the difference?
https://code.claude.com/docs/en/chrome