browser sessions start failing at around 20 concurrent. nobody warns you about this
Summary
Playwright scrapers in production on Node.js start failing around 20 concurrent browser sessions, causing memory spikes and crashes. The developer notes documentation does not warn about this limit.
Similar Articles
@browser_use: Here's 25 browsers starting in less than 1 second Enjoy
Browser Use launches a new browser infrastructure service featuring subsecond cold starts, lower cost at $0.02/h, and unlimited scaling, now live for developers.
stale html and headless browsers kept getting me blocked, so i started replaying the actual requests instead
The author shares their experience of switching from headless browsers to replaying direct requests to scrape websites, reducing block rates and resource usage significantly.
40% of my browser agent's sessions were silently failing and the LLM wasn't the problem
A developer discovered that 40% of browser agent sessions silently failed due to browser fingerprinting and automation detection, not LLM reasoning. An open-source tool called Leakish identified the issues.
@larsencc: https://x.com/larsencc/status/2053862900289470765
This article details the production architecture behind the open-source browser-use library, explaining how to scale browser agents using AWS Lambda, SQS, and S3 for state management and retries.
Built a Playwright version that lets AI-Agents navigate the web
A fork of Playwright that generates unique browser fingerprints per session to enable AI agents to navigate the web undetected. The project is fully open-source under MIT license.