SEO

Wikimedia's own engineering blog reported a 50% bandwidth surge from AI scraping, with bots making up 65% of the most expensive traffic while being a third of pageviews. Smaller sites are reporting the same pattern. Here's what a normal site can actually do about it.

In August 2025, Cloudflare accused Perplexity of disguising its crawler to keep scraping sites that had blocked it. Perplexity disputes the characterization. Here's what Cloudflare actually documented.

Nobody can promise you'll get more ChatGPT or Perplexity referrals. But several named analytics firms independently measured what that traffic does once it arrives — and it converts meaningfully better than organic search, even though the exact multiple depends on who's counting.

GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider and a dozen more — what each one actually does, who runs it, and the robots.txt rule that matches what you're trying to opt into or out of.

Three products, three different audiences, three different ways of deciding what to cite. A founder with limited time needs to know which one actually rewards the effort before spending any of it.

Google has said plainly it doesn't use llms.txt. A 137,000-domain study found 97% of published files get zero requests, ever. We still publish one — here's why that isn't a contradiction.

A curl check tells you what a crawler receives once. It tells you nothing about whether GPTBot, ClaudeBot or PerplexityBot actually show up on a Tuesday in October. Here's how to monitor that on an ongoing basis.

There is no special 'AI schema.' Google's own documentation says AI Overviews eligibility rides on the same requirements as an ordinary search snippet — but that doesn't mean structured data is pointless, and it doesn't mean FAQ markup gets you cited either.

Blocking GPTBot opts your site out of future OpenAI training runs. That's the whole effect. It does nothing to whether ChatGPT can cite you — that's a different crawler entirely.

One Princeton study measured a 22-41% visibility lift from adding quotes, statistics and citations inside a lab benchmark. It's the most solid data that exists — and it isn't the same claim as 'do this and ChatGPT will mention your product.' Here's the actual gap.

Wikipedia and Reddit together account for more than a quarter of ChatGPT's US citations, per a 5W synthesis of 680 million tracked citations. But the two sources aren't interchangeable — and neither is reachable on demand.

The research on AI citation says quotable, specific claims get cited more than vague ones. This is the practical follow-up: how to actually structure a paragraph so it can be lifted out of your page and still be correct.

Most 'best SaaS directories' roundups quote a domain-rating estimate and call it a link audit. We read the rel attribute off a real listing page on each of these 18 instead.

The badge is now the usual price of a free dofollow directory link. Here are 28 confirmed dofollow directories that don't charge it — no badge, no reciprocal link, just a listing.

Submitting to a directory is simple. Doing it in a way that doesn't waste the afternoon, or quietly lose the link you thought you'd earned, is not.

A launch board gets you a day of traffic. A review platform gets you found six months later by someone comparing options with a credit card in hand. Both jobs are worth doing — they are not the same job.

A week-one sequence for a founder who just shipped, has no budget, and wants real dofollow links now — what to do first, what to trade a badge for, and what to skip.

Most 'best directories' lists assert dofollow with no evidence. We read the rel attribute on every directory we could reach: 34 confirmed dofollow, 2 confirmed nofollow, 25 that block the check.

Googlebot renders JavaScript. The crawlers behind ChatGPT, Claude and Perplexity do not. A client-rendered site can rank perfectly well in search and still arrive at an assistant as an empty shell.