LinkedIn blocks AI crawlers like GPTBot and ClaudeBot but allows search-index bots such as OAI-SearchBot, providing limited profile data that omits key information like job titles and experience dates.
No links in this post. Everything below is reproducible with one curl per line. I fetched the same LinkedIn profile URL six times, changing only the User-Agent. GPTBot -> HTTP 999 ClaudeBot -> HTTP 999 ChatGPT-User -> HTTP 999 Googlebot -> HTTP 999 OAI-SearchBot -> HTTP 200 Claude-SearchBot -> HTTP 200 999 is LinkedIn's block code. The training crawlers are refused, the user-triggered fetchers are refused, and the two search-index bots are let in. Googlebot is refused as well, almost certainly because LinkedIn verifies it by reverse DNS and a spoofed UA fails that check. The AI search bots do not appear to be verified at all, which means the UA string alone is enough to get the page. Then I parsed what the 200 actually contains, on an ordinary mid-career profile rather than a celebrity one. The JSON-LD graph has a WebPage node and four DiscussionForumPosting nodes. There is no Person node. The WebPage node is two fields, a name and a URL. In the rendered markup: no jobTitle anywhere no About section at all Experience renders the company name with no role and no dates Education renders the school name with no dates no skills, no certifications, no recommendations The four post nodes do carry full body text. The newest is December 2024. The one before it is a July 2023 hiring post for a product that person no longer works on. For contrast I ran a Creator-mode public figure. That profile does emit a Person node, with alumniOf carrying start and end years and worksFor carrying one company name. But its jobTitle field comes through as five empty strings, five roles and no titles, and the description is truncated mid-sentence. The part I find interesting is not the blocking. It is the shape of what gets through. A model doing retrieval on a person gets a name, a location, one employer name, a school name, and whatever that person happened to post publicly, which for most people is old and unrepresentative. Everything that would actually answer "what does this person do" is either absent or an empty string. So when an assistant answers a question about a working professional, it is not reasoning over a profile. It is reasoning over a company name and some three-year-old posts, and filling the rest in. Two things I cannot resolve from outside: Whether the empty jobTitle is deliberate policy or an artifact of how the logged-out page is assembled. Every profile I checked behaves the same way, so I lean policy, but I cannot prove intent. Whether adding a crawlable page elsewhere actually displaces the stale sources in a model's answer, or merely joins them. That is a ranking question inside retrieval and I have no way to measure it. If anyone has run a controlled before and after on the second one, I would rather read that than keep speculating.
LinkedIn has implemented a dynamic behavioral scoring system that replaces the fixed connection request cap, using factors like acceptance rate and SSI to determine account trust and volume limits, impacting AI automation tools like Expandi.
A LinkedIn user hid a prompt injection in their bio, causing AI-driven recruitment bots to respond in Old English and address them as 'My Lord', demonstrating the manipulability of AI agents.
Grok Bot, now in early beta, is an AI teammate that can sign into tools and complete tasks like finding LinkedIn candidates based on natural language requirements.
A study by AI detection company Pangram found that 41% of longform posts on LinkedIn are fully AI-written, with X at 25%, while Reddit and Substack show much lower rates around 10%, highlighting platform differences in AI content prevalence.
Analysis of 11 million crawler logs across 34 websites reveals distinct behaviors: GPTBot crawls relentlessly ignoring robots.txt, Google's bot checks rules frequently, ClaudeBot's crawling is rapidly accelerating, and Bytespider is the heaviest crawler. The findings suggest a shift from Google-centric SEO to optimizing for AI agent page selection.