@AmberTreelet: Tiance Ge shared yt-dlp for scraping Douyin, YouTube, Bilibili, Twitter. I'll add some universal scraping tools. FxTwitter: Recommended by @0xCheshire for scraping X. get笔记 (Dedao Brain): WeChat Official Accounts, Xiaohongshu, Douyin, Bilibili, X, Podcasts. Google Chrome extension obsidian web clipp…
Summary
Introduces multiple web scraping tools, including yt-dlp, FxTwitter, get笔记, etc., for scraping content from different platforms.
View Cached Full Text
Cached at: 06/20/26, 04:20 PM
Tiance shared yt-dlp for scraping Douyin, YouTube, Bilibili, and Twitter.
Let me supplement with some universal scraping tools:
- FxTwitter: Recommended by @0xCheshire for scraping X (Twitter)
- get笔记 (Dedao Brain): Works with WeChat Official Accounts, Xiaohongshu, Douyin, Bilibili, X, and podcasts
- Google Extension - Obsidian Web Clipper: Scrape content and save it to Obsidian
- Google Extension - YouTube to Notebook LM: Great tool for YouTube
- GitHub’s Agent Reach + Google Extension Cookie-Editor: Supports over a dozen platforms; for restricted ones, use the extension to export cookie data
- Snipd: AI-powered podcast clipping tool
Similar Articles
@NFTCPS: Finally found out where those repost accounts on X get their content! It's this tool MediaCrawler, a single tool that covers Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, and Zhihu. It can scrape public content, comments, likes, and reposts. The best part is it doesn't need JS reverse engineering—it uses browser login state to get signatures directly, …
MediaCrawler is a multi-platform social media data scraping tool that supports public content crawling from Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, and Zhihu. It bypasses JS reverse engineering by leveraging browser login state, lowering the technical barrier.
@gkxspace: Get notes is really incredible. So far I haven't found anything it can't capture. I've tried various domestic and overseas platforms: Xiaohongshu, Bilibili, Douyin, WeChat Official Accounts... TikTok, YouTube, Twitter... Just give it a link, and my success rate is 100% so far. Also, you can directly use the official skills to connect it to …
The Get Notes tool can capture content from major domestic and overseas platforms (Xiaohongshu, Bilibili, Douyin, YouTube, Twitter, etc.) with a 100% link capture success rate. It also supports connecting to tools like Codex via official skills.
@xiaoerzhan: The 'Xiaoer Grab Video' browser extension I use daily is now open-source. One click and the current page's video is downloaded locally. Supports 1800+ sites like YouTube, Bilibili, X, TikTok, etc. This tool was created because: I believe video tutorials are the most effective – you follow along step by step, the visuals are clear. But then the things I want to learn keep piling up…
The author announces that the daily-use 'Xiaoer Grab Video' browser extension is now officially open-source. It supports downloading videos from 1800+ sites including YouTube, Bilibili, X, TikTok, etc., and provides a complete pipeline of converting videos to text and building a knowledge base.
@WY_mask: MediaCrawler: Open-source web scraping tool for Xiaohongshu, Douyin, Weibo, Bilibili, Kuaishou. Supports scraping videos, images, comments, likes, reposts, etc. https://github.com/NanmiCoder/MediaCrawler…
MediaCrawler is an open-source multi-platform self-media data collection tool that supports scraping public information from Xiaohongshu, Douyin, Weibo, Bilibili, Kuaishou and other platforms. No JS reverse engineering required, based on Playwright browser automation.
@gengdaJ: Stunned, this TikTokHub site is absolutely mind-blowing... It has social media APIs for all platforms at home and abroad. Twitter API costs less than one cent per call, and even Xiaohongshu, which has always been the hardest to deal with, is only 7 cents. TikTok, Reddit, WeChat Official Accounts, Douyin are a piece of cake. This is now on my core tool list...
Recommend a site called TikTokHub that provides social media APIs for all platforms globally at extremely low prices, covering Twitter, Xiaohongshu, TikTok, Douyin, Reddit, etc. It can be used for data scraping, monitoring, topic collection, and more.