There is a failure mode that does not show up in any of your dashboards. Your site ranks. Analytics look normal. You open the page in a browser and it is obviously fine. And an assistant, asked about the exact thing you built, has nothing to say about you — not because it judged your product, but because it never received any words.
The cause is unglamorous. Googlebot executes JavaScript. The crawlers behind ChatGPT, Claude and Perplexity do not. Vercel and MERJ measured this across more than 500 million GPTBot fetches and found no JavaScript execution at all. So a single-page app that assembles its content in the browser is two different documents depending on who asks for it: a full page for Google, and whatever sits in the initial HTML for everyone else.
If that initial HTML is a <div id="root"> and a script tag, that is the whole
site as far as an assistant is concerned.
The same URL, two different documents
This is worth being precise about, because "AI can't read my site" gets said in a vague way that leads people to buy the wrong fix.
Nothing is being blocked, hidden or penalised. The crawler makes an ordinary HTTP request and gets an ordinary 200 response. It simply keeps what the server sent and stops there. Google's crawler does a second pass and runs the scripts; the AI crawlers have no second pass.
The practical consequence is that every rendering choice you made for performance or developer convenience now has a distribution consequence you were not thinking about. Content behind a client-side fetch, a tab that loads on click, a pricing table hydrated from an API, a description rendered by a framework with no server pass — all of it is present for a human, present for Google, and absent for an assistant.
What "invisible" actually looks like
Three shapes, in rough order of how often they turn up.
The empty shell. The server returns 200 and about forty words: a title, a noscript line, maybe a nav. Everything that explains the product arrives later, in JavaScript. This is the classic client-rendered app, and it is the hardest to notice because nothing anywhere reports an error.
The 403. Bot protection at the CDN or firewall refuses the crawler's user
agent before your application is ever reached. Your robots.txt says the
crawler is welcome; your edge disagrees, and the edge wins. Nobody chose this —
it usually arrives as a default when someone turns on bot filtering.
The accidental robots rule. Someone added a blanket disallow to keep the site out of AI training and took out the search crawler in the same line. Which brings up the distinction almost everybody gets wrong.
GPTBot and OAI-SearchBot are not the same bot
GPTBot collects training data. OAI-SearchBot fetches pages for the index
behind ChatGPT's search citations. They are independent, and blocking one has
no effect on the other.
Block GPTBot and you have opted out of training runs. That is a legitimate
choice and it costs you nothing in ChatGPT's answers. Block OAI-SearchBot and
you are removed from those answers entirely — you cannot be cited from an index
you are not in.
A blanket User-agent: * disallow aimed at "AI" does the second thing while
people believe they are doing the first. If you have ever pasted an AI-blocking
robots snippet from a blog post, this is worth thirty seconds of your time.
We hit the same wall from the other side
When we checked 51 startup directories for dofollow links, our
checker fetched a real listing page and read the rel attribute off the
outbound link. Most of what we could not verify was bot protection returning
403 — the second shape above, aimed at us.
But two directories defeated the check for the first reason: they render their outbound links client-side. The anchor does not exist in the HTML that the server sends. Our dofollow checker sees no link at all, and neither does anything else that does not run a browser.
Those directories are almost certainly fine for humans. The point is narrower and worth sitting with: if a link only exists after JavaScript runs, then every consumer of your page that does not run JavaScript is working from a document with no link in it. The same logic that hides your product description from an assistant hides an outbound link from a crawler.
Check your own page in two minutes
You cannot see any of this in a browser, because your browser is the one thing in this story that definitely runs JavaScript. You have to fetch the page the way a crawler does.
The one-liner:
curl -sA "OAI-SearchBot" https://yoursite.com | wc -c
curl -sA "OAI-SearchBot" https://yoursite.com | grep -c "your product description"
If the byte count is small and the grep returns 0, the text is not in the
document. Repeat with ClaudeBot and PerplexityBot, because a block can be
per-agent, and check the status code separately — curl -o /dev/null -w "%{http_code}"
will tell you whether you are looking at an empty shell or a refusal.
We built the AI crawler check to do this without the terminal: it fetches your page as each crawler, once, and shows the status code each one got and the text each one received. It is free and there is nothing to sign up for.
Fixing it is a rendering change, not an SEO change
If the text is missing, the fix is to make the server send it. Server-side rendering, static generation, or prerendering for bot user agents — any of the three works, and which one you pick is an ordinary engineering decision about your stack, not a marketing one.
Two things that do not fix it: adding structured data (it is markup about content, and it is usually injected client-side too, in which case it is equally invisible), and writing more content (content the crawler cannot see does not get better by being longer).
If the problem is a 403, the fix is in your CDN or firewall rules, and the
tell is that robots.txt says one thing and the response code says another.
Be honest about what this buys you
Getting readable is necessary. It is not sufficient, and the gap between those two words is where most of the AI-visibility industry currently lives.
Being readable means an assistant can receive your page. It does not mean an assistant will mention you. We know of no evidence that any particular action reliably causes an assistant to recommend a product, and there is a useful counterexample: G2 and Capterra carry enormous authority and get near-zero citations. Authority alone plainly does not convert into being named.
Directory listings, ours included, have not been shown to cause AI citations either. What is observable is weaker and honest to state as such: consistent descriptions of a product across independent pages correlate with that product being named by assistants. Correlation is the whole of the claim. Nobody has shown causation, and anyone selling you AI visibility as a deliverable is charging for a mechanism that has not been demonstrated.
It is also harder to measure than it looks. Asking a model "do you know this product" and writing down the answer is not measurement — between 9% and 28% of such decisions flip on repeat runs, even at temperature zero. A one-shot check presented as a score is a coin toss with a progress bar attached.
So the sequence that survives scrutiny is short. Make sure the crawler can reach your page and receives the words on it, because that part is mechanical, verifiable and identical on every run. Then describe your product the same way everywhere it appears — your own site, your docs, and whatever independent pages list it, including a LetsLaunch listing if you want one more. Then stop, because past that point nobody can currently tell you what works.
Does ChatGPT render JavaScript?
No. The crawlers behind ChatGPT, Claude and Perplexity execute no JavaScript — Vercel and MERJ confirmed this across more than 500 million GPTBot fetches. They read the HTML your server sends and nothing more. Googlebot does render, which is why a client-rendered site can rank in Google while arriving at an assistant as an empty page.
Will blocking GPTBot remove me from ChatGPT's search results?
No, and this is the most common confusion. GPTBot collects training data
while OAI-SearchBot fetches pages for the index behind ChatGPT's search
citations. They are separate crawlers, so blocking GPTBot keeps you out of
training runs without affecting search. Blocking OAI-SearchBot is what
removes you from ChatGPT's answers, and a blanket disallow rule does that by
accident.
Why does my site rank in Google but not appear in AI answers?
Usually because Googlebot renders JavaScript and AI crawlers do not, so the two are reading different documents at the same URL. Fetch your page with a crawler user agent and no browser to see which one you have — an empty shell means a rendering problem, and a 403 means a bot-protection rule. Neither shows up in your analytics, because a crawler that receives nothing has nothing to report.