I tested how detectable 23 browser agents are from the website's side. Only one signs its requests
Summary
作者对 23 个浏览器 AI 代理进行了网站侧可检测性测试,发现只有 Manus 正确使用了 Web Bot Auth 签名标识自身,而多数代理(包括 ChatGPT 的 agent)的请求未签名且难以与人类区分,仅靠指针移动轨迹即可识别所有点击型代理。
Similar Articles
I opened my small website to AI agents and logged who showed up. Mostly it’s bots indexing bots.
一位小网站站长开放了对 AI 代理的支持(llms.txt、Markdown、MCP 服务器、按次付费端点)并记录流量:绝大多数是 Meta、Amazon 等 AI 爬虫以及目录和扫描类 MCP 客户端,真正干活的代理调用仅有 3 次,说明当前的“代理互联网”主要是机器人在索引机器人。
@GitHub_Daily: AI agents automating browser operations or scraping data often get blocked by anti-scraping mechanisms, and get stuck when encountering captchas or human verification. Recently, the BrowserAct team open-sourced a Skill, a browser automation command-line tool designed specifically for AI agents. It provides three layers of anti-blocking mechanisms, from…
The BrowserAct team open-sourced a browser automation command-line tool designed specifically for AI agents, providing three layers of anti-blocking mechanisms (fingerprint spoofing, captcha cracking, human takeover), supports multi-browser parallelism and account isolation, and optimizes output format to save tokens.
@LukeberryPi: how to give agents control of your browser? - playwright mcp - browser-use - agentbrowser what else?
一条推文探讨如何让 AI 代理控制浏览器,列举了 Playwright MCP、browser-use、agentbrowser 等工具,并询问还有哪些替代方案。
Giving all my AI agents one shared browser instead of a login each
作者分享为多个 AI 代理(Claude、Claude Code、ChatGPT)搭建一个共享云端 Chrome 浏览器的做法:统一登录、隔离标签页、敏感操作人工审批、保留可读日志,并将可重复流程录制回放。该方案已集成进其创办的工具 Daiku(目前仅限邀请)。
Each AI agent crawls website completely differently. Here's what 3 mons of 11 million event logs actually show.
Analysis of 11 million crawler logs across 34 websites reveals distinct behaviors: GPTBot crawls relentlessly ignoring robots.txt, Google's bot checks rules frequently, ClaudeBot's crawling is rapidly accelerating, and Bytespider is the heaviest crawler. The findings suggest a shift from Google-centric SEO to optimizing for AI agent page selection.