scraping

A tweet shares DeepAPI benchmark results claiming Opus 5 outperforms GPT-5.6 Sol by 69% at creating web search queries, winning all 53 blind comparisons.

A judge rejected Google's defense under the DMCA to avoid liability for being scraped, a significant ruling for web scraping and copyright law.

Reddit now requires login to use its old design (old.reddit.com), citing safety concerns about abusive scraping and automated traffic, which frustrates users who prefer the simpler interface.

Olostep is a web data API that enables extraction, crawling, and structuring of web data at scale, designed for AI teams, data pipelines, and automation.

Patreon has partnered with Cloudflare to actively block AI bots from scraping creator content for training, moving beyond the voluntary robots.txt approach. The move aims to give creators more control over how their work is used by AI companies.

A hack reveals that AI music generator Suno trained its models by scraping millions of songs from YouTube, Genius, and Deezer, backing up allegations of copyright infringement and exposing customer data.

A hack into AI music generator Suno reveals it allegedly scraped decades of audio from YouTube, Deezer, and other sources for training data, raising copyright and DMCA concerns amid ongoing lawsuits from major record labels.

A user praises a data scraping tool, saying that aside from the occasional need to update the Xiaohongshu API, it basically meets all scraping needs.

shot-scraper 1.11 is released with minor improvements such as a longer wait time for server processes and new command options like --js-file and --timeout for consistency.

Introduces a new yichen-skills tool wechat-mp-batch-exporter that allows using Codex to automatically batch scrape WeChat official account articles, including text, likes, shares, comments, read counts, and other data, with login status maintained for 4 days.

Fortress is a stealth Chromium engine that modifies browser fingerprints at the C++ level to help scrapers and browser agents avoid detection by bot detectors like Cloudflare Turnstile, CreepJS, and Sannysoft. It operates as a drop-in CDP replacement for Playwright and Puppeteer.

TikHub is an API aggregation station for social platform data, supporting 16 platforms (such as Douyin, Xiaohongshu, TikTok, etc.), providing over 1000 APIs for retrieving videos, comments, user profiles, e-commerce data, and more.

This article critically examines the accuracy of AI visibility tools that claim to measure brand presence in generative AI responses, arguing that they provide false precision due to nondeterminism, personalization, and scraping biases. It calls for transparency in methodology and warns against treating opaque dashboards as stable truth.

Introduces ScrapeGraphAI, an LLM-based scraping tool that can take natural language descriptions of requirements and automatically generate the scraping workflow, eliminating the need to write selectors or care about HTML structure, supporting local models and multiple integration platforms.

Reddit will require users to log in to access old.reddit.com, citing the need to combat abusive scraping and automated traffic, which may upset users who prefer the old interface for its simplicity and privacy.

Residential proxies, used for scraping and hiding nefarious activity, are rising as bots now generate more internet traffic than humans. The article explores the technical and ethical challenges, including payment protocols like Cloudflare and Coinbase's x402 standard.

A blazing-fast, stateless CLI tool written in Go that integrates Web search, code search, and library documentation query. It supports web scraping and site crawling, designed for AI agents and terminal use.

35 newspaper publishers across the US have filed a lawsuit against OpenAI and Microsoft, alleging that the companies scraped their copyrighted and paywalled content without permission to train ChatGPT, harming local journalism.

A detailed guide on building an AI-powered content machine that scrapes viral content from platforms like TikTok and Instagram, uses AI to generate platform-native posts, and automates scheduling with tools like ScrapeCreators, Kie.ai, and Postiz.

Playwright scrapers in production on Node.js start failing around 20 concurrent browser sessions, causing memory spikes and crashes. The developer notes documentation does not warn about this limit.