Step-by-Step: Build Your AI Topic Pipeline — From Information Sources to Topic Database

Honestly, about three months ago, the thing I dreaded most every day was coming up with article topics.

First thing in the morning, I’d scroll through X. Spend half an hour watching tons of people posting content. I wanted to post something, but my mind was completely blank. The more I scrolled, the more anxious I got. Eventually, I wasn’t even reading content — I was just mindlessly swiping.

When I finally found something I thought I could write about, I’d open the editor. Write three lines, delete them. Write two more, delete them. Then close it and say, “I’ll do it tomorrow.”

Then I stopped doing that. I built a system. It runs automatically every morning at dawn. I open Obsidian, and there are already 10 topic ideas waiting for me, complete with source links, data, and angles. I spend 5 minutes scanning through them, and I know what I’m writing that day.

It’s been running for three months, iterated through several versions. Today I’m sharing the entire method publicly. 🧵

Let’s start with something fundamental.

Many people think topics rely on inspiration. When inspiration strikes, words flow. When it doesn’t, you just wait.

My biggest takeaway from these months: Topic selection is not an inspiration problem — it’s an information processing problem.

Think about how much information you see every day: X, WeChat official accounts, news, group chats. The human brain can’t process it all. When you can’t process it, you get anxious. When you’re anxious, you scroll mindlessly. Scrolling mindlessly makes you more anxious.

Hand the “information hunting” task over to a system. Save your brainpower for making judgments. That’s the right approach.

Alright, here’s exactly how to do it. Five steps.

Step 1: Define your water sources.

Topics don’t come out of thin air. They’re filtered from information. Your information sources are your water sources. If the water is dirty, everything downstream is wasted.

I divide them into four layers.

First layer, fast water. This is X trends, Weibo hot searches, tech news. Check every few hours to make sure you don’t miss something brewing.

Second layer, deep water. 36Kr, Huxiu, Hacker News, GitHub Trending. Check once a day. These sources have substance — not recycled gossip passed through multiple hands.

Third layer, slow water. Industry reports, deep analyses, competitor updates. Check two or three times a week. Topics from this layer don’t go stale quickly and are good for long-form content.

Fourth layer, reverse water. This is the most overlooked: comment sections. Not for arguing — to see what people are asking. A question someone chases in a comment thread is something at least 100 other people also want to know. A question is a topic.

Today, do one thing: list 3 sources for each layer. Don’t overthink it. Three is enough.

Step 2: Cast your net.

Once you have your water sources, you need to tell AI what you care about. Keywords are your fishing net.

For example, I follow AI Agents. My keywords look something like this:

AI Agent frameworks, multi-agent collaboration, Agent memory
LangGraph, MCP protocol, A2A
Agent hallucination, Agent timeout, Agent cost

One important point: don’t just write industry terms — make sure to include pain-point keywords. Things you find with pain-point keywords are naturally topic-worthy, because “pain” equals reader interest.

This isn’t a set-it-and-forget-it thing. Every Sunday night, I spend a few minutes reviewing. Delete outdated terms, add whatever emerged that week.

Step 3: AI initial screening.

Water sources ready. Net ready. Let AI go to work.

In my setup, it runs automatically every morning. AI searches 5 to 8 keywords in parallel, returns a few results per keyword. Deduplicates, sorts by timeliness, prioritizes items from the last 24 hours. Then it fetches the full content and reads it.

After that, a topic briefing is automatically written into my Obsidian. It includes key findings, critical data, and source links. I open it in the morning and read directly.

Let me address a common pitfall.

Many people ask AI to directly generate topics. That approach is wrong.

What AI should do is information compression: compress 100 articles into 10 summaries. AI is not responsible for deciding which one is worth writing.

Judgment is the human’s job. A human reads the summaries, uses experience and intuition, and picks what’s worth writing. AI handles speed; humans handle accuracy.

Step 4: Human judgment.

The briefing is in hand. How do I review it? I ask myself four questions for each item.

First, is the angle fresh? Search for it: if the top 10 results all take the same angle, it’s not fresh. If others have already beaten it to death, pass.

Second, do I have unique information? First-hand experience, exclusive data, something only you can write. If so, this topic is solid.

Third, does the reader care? Would your target reader be willing to spend a few minutes reading this?

Fourth, can it grow followers or generate actual value?

After answering these four questions, the verdict is pretty clear. I tag each topic. The highest-level tag is for one specific situation: fresh angle and information nobody else has. If you find two or three such topics a week, you’re doing great.

Let me pause here and add something about copywriting formulas.

A lot of people ask me: what about those PAS, QUEST, SCAR formulas — how do you actually use them?

Let me tell you: formulas aren’t just for when you’re writing. You should use them during the topic selection phase.

When you get a topic, think: what formula fits?

For example, a topic about something that makes people anxious — use PAS. Problem, Amplify the pain, Solution. If you fill in the three boxes with material at the topic stage, the article skeleton is already standing.

For a teaching-oriented piece, use QUEST. Question, Empathize, Explain, Stimulate, Transition. Fill in the five nodes, and you have a framework.

For a true personal experience, use SCAR. Story, Conflict, Aha moment, Resolution. Real experiences are naturally compelling; the formula helps you organize that impact.

Spend 30 seconds at the topic stage figuring out what structure to use, and you won’t find your structure collapsing halfway through the writing.

Step 5: Queue and schedule.

After tagging, throw the topics into your database.

The highest-grade ones: write and publish the same day. Good ones that need more digging: publish within 3 days. The rest — common topics — keep in the database. When you have nothing to write one day, go dig them out.

For each topic, I also record a few things: three to five alternative titles, the core angle, three to five key pieces of material, and which platform to publish on.

But don’t settle on the title yet. This is a hard rule: write the body first, then decide the title. You need to write the body to know what you’ve actually delivered. Often, the topic you thought you had turns out to have a completely different core argument by the time you finish. The hook is always written last.

After running this system for three months, what’s the most obvious change?

Before, relying on inspiration had one huge problem: instability. One day I’d have inspiration and write three articles. Then for the next three days, I couldn’t squeeze out a single word.

With a system, every morning I open Obsidian and the topics are already there. I only have one thing left to do: choose.

It’s not that my judgment got sharper. It’s that I offloaded the “finding information” part to the system, so my brain only handles judgment. The human brain is simply not designed for information retrieval. It’s designed to look at 10 summaries and intuitively pick the best three.

Finally, here are three things I realized over these months.

One, the most searched topic is often the least valuable. Everyone is searching for it, everyone is writing about it. You can’t compete in that race.

Two, the most underestimated topic source is the questions other people ask. Not hot searches, not headlines — just a plain question in a comment section. One follow-up question is one topic.

Three, content about your own mistakes always spreads better than content about your successes. Success may not be replicable; failures are always relatable. My best-performing articles were all about things I screwed up.

Things you can do right now.

Spend 10 minutes listing your four layers of water sources: three sources per layer. Don’t overcomplicate it — just write down what you already browse.

Then create a keyword list, under 20 items. Pain-point keywords should make up at least half.

Finally, use any AI tool you already have. Give it your sources and keywords, and let it run one search-and-summary session. The first run will probably be terrible. That’s fine. After that run, you’ll know what to adjust, and the next time will be much better.

Once you’ve built your pipeline, it’s 5 minutes every morning, and 10 topic ideas are in hand. You never have to worry about this again.

What’s next: now that you have topics, how to write a headline that makes people want to click. Follow me, update coming soon.

📚 Archive of Previous Articles

• Super Virtual Team: A Practical Guide to Multi-Agent Collaboration

• AI Is Increasing Your Startup Failure Rate

• I Hand-Built a Chrome Extension to Batch Organize My X Bookmarks into an Obsidian Knowledge Base

• DeepSeek’s One JD Is a 2026 AI Career Manual

• Managing 10 AI Employees for 3 Months: 5 Hard-Earned Rules After Falling into 234 Pits

• An Old Newbie’s X Account Pitfall Postmortem: I Almost Ruined My 16-Year-Old Account

• Don’t Treat Codex as Just a Code Assistant — It’s Becoming a Workflow System

• How to Use Codex’s Pinned Threads Effectively

• Codex App No-Nonsense Getting Started Guide: These Few Commands Are Enough

If this article helped you, feel free to follow + bookmark + share 👏🏻

Follow @thinkszyg for continuous sharing of real-world, production-level AI dry goods.