Tag
博主分享如何成功让自己的网站被 Google 除名（deindex），以应对 AI 抓取带来的内容被肆意爬取的问题，并解释了 robots.txt 与 meta robots 标签配合使用的技巧。
Reddit will shut down RSS feeds on November 13, 2026 and end public API access by March 2027, citing large-scale scraping and AI bots, pushing tools and AI assistants toward commercial data deals. Moderators must migrate workflows to the Discord Relay app, and Old Reddit will be restricted to logged-in users to curb abusive traffic.
Reddit is further restricting access to Old Reddit, requiring recent usage within six months, and will drop RSS feed support on November 13th to combat AI bots and automated scraping.
A non-technical individual questions whether AI systems scrape illegal content like child pornography and if server owners face legal liability for possession.
Internal documents from Microsoft and OpenAI reveal executives' concerns that AI scraping of news content is the 'largest theft of labor in human history' and could create a 'doom loop' harming news organizations and model performance.
New unredacted filings in a lawsuit reveal that a Microsoft executive described AI scraping as 'the largest theft of labor in human history,' with internal admissions about AI's impact on publishers like The New York Times.
An individual scraped all public art from the Cara platform, an anti-AI art community, and after facing backlash, is now collaborating with the platform's founder to develop an open-source tool to protect artists from AI scraping.
A new font called ShieldFont leverages ligatures to replace words in webpage HTML, presenting readable text to humans while serving nonsense to AI scrapers, effectively poisoning training data.
Gamers Nexus announces it is starting to poison its free benchmark charts to deter AI scrapers, with subtle pixel-level data manipulations tested against LLMs, as bot traffic now exceeds human traffic online.
Reddit's DMCA lawsuit against SerpApi and Perplexity AI over Google search scraping continues, with a judge allowing discovery despite dismissing some claims. The case highlights ongoing tensions between AI scraping and copyright law.
Patreon has partnered with Cloudflare to actively block AI bots from scraping creator content for training, moving beyond the voluntary robots.txt approach. The move aims to give creators more control over how their work is used by AI companies.
Codex, an AI model, scraped the ICM website and discovered a hidden list of 2026 Fields Medal winners, including Peking University alumni Hong Wang and Yu Deng.
An update on the escalating problem of AI scraper bots overwhelming websites, discussing residential proxy networks and their impact on the open web.
Anthropic accused Alibaba of creating tens of thousands of fake Claude accounts for distillation attacks, leading to stricter guardrails that are affecting legitimate users.
The article discusses a website using Anubis, a proof-of-work system, to protect against mass scraping by AI companies, highlighting the ongoing battle between content providers and AI data collection.
Strava is tightening its API access by requiring a paid subscription for developers, citing a 448% increase in developer applications due to zero-code AI tools and scraping attempts that degrade platform performance.