Everyone writing about SEO now has a second acronym to sell you: GEO — generative engine optimization, getting your content surfaced when someone asks an AI assistant instead of typing into a search box. The advice is abundant and reasonable: FAQ schema, a question-and-answer voice, clean canonical HTML. What almost nobody does is check whether any of it worked. You can pour weeks into being AI-legible with no idea whether a single assistant has ever surfaced you to a single person. That is not optimization; it is superstition with a publishing calendar. Here's the cheap harness I built to turn the question into a number I can watch.
The gap persists because the question feels unanswerable: the assistants rarely tell you when they've quoted you, so people fall back on proxies and assume it's working. The reframe that makes it tractable is to ask a narrow, repeatable question instead: when a potential user asks the specific thing my content answers, does an AI cite me? Fix the questions, fire them weekly, record whether your site is in the answer's sources, and the citation rate becomes a scorecard — and a falsifier: a week of "optimization" that leaves the rate flat three weeks later did nothing, and now you have the receipt.
The queries are the heart of it, and there are two rules. First, they're fixed: a trend only means something if the measuring stick doesn't change, so swapping questions throws away comparability. Second, each is phrased as a sentence a real user would type, not your brand name — "Is there an MCP server for searching Korean patents?", not "korean-patent-mcp." Prompting the AI with your name to check it knows your name is vanity; what drives traffic is whether it surfaces you when someone describes the problem you solve. Each question is paired with the owned patterns that count as a hit in the answer's sources:
{
"id": "agent-file-work-bus",
"query": "How do I orchestrate multiple AI agent CLIs without a message broker?",
"assets": ["yoursite.com/notes/file-based-agent-work-bus", "yoursite.com/notes"]
}
When you publish a note, append its question and owned patterns; the probe watches for it next week.
The match has to be deterministic — a plain substring test, never a model deciding "were you cited?" The moment an LLM grades its own citation, it stops being reproducible. So the rule is boring on purpose: does one of my owned patterns appear in a citation URL, or in the answer text? A URL hit is a strong signal (it linked me); a prose hit is weak (it named me without linking). Keep both, never conflated.
The correctness fix that mattered most was scoping every pattern to a path I own. My first version matched bare names, and a repo of mine shares its bare name with a stranger's same-named clone — so the probe counted that stranger's project as "me" — citations I never earned. A bare patent-mcp is anyone's; owner/patent-mcp is mine.
owner/repo, App Store ID — so a "hit" can only ever be you.You can't measure this with an ordinary chat model: a plain LLM either answers from stale weights with no live retrieval, or won't expose its sources — either way there's nothing to check. The probe needs a grounded model that retrieves live results and returns the URLs it used. I use Perplexity's Sonar through OpenRouter: an ordinary chat-completions endpoint that returns the citation URLs as structured annotations — the entire measurable surface.
The economics make it a habit. Sonar runs about half a cent per query; a couple dozen questions once a week is roughly a dime — under a dollar a month. Cap the answer tokens low — you want the sources, not the prose — and space the calls so a rate limit doesn't truncate the sweep. The budget is never the constraint; discipline is.
Here's the bug that silently wrecks the trend: a failed round is not a citation rate of zero. The first time I turned the probe back on, every query failed with HTTP 402 — the key was valid but the account was out of credit. Had the probe logged those as "not cited," the ledger would show a cliff: a collapse to zero across every query, indistinguishable from "the AI stopped citing me." A fabrication caused by an empty wallet.
The fix is to make errors a distinct record. A failed query writes an error row, not a cited: false row, and every summary and trend skips error rows entirely:
{"ts": "...", "id": "agent-file-work-bus", "cited": true, "matched": ["url:yoursite.com/notes/file-based-agent-work-bus"], "n_citations": 12}
{"ts": "...", "id": "agent-file-work-bus", "error": "HTTPError"}
A 402 sweep then reads as no data this week — the truth — not a decline. The probe fails loudly to stderr so I notice, but keeps going through the rest of the set. The principle outlives the tool: a gap in the record must never masquerade as a measurement.
The other way to get a garbage number is to measure too early. Citations don't appear at publish; they follow a pipeline — publish → crawl → rank → cite — and each stage takes time. Measuring three days out records a wall of zeros that mean "too early," not "not working" — confuse the two and you'll panic or write off good content.
I walked into this. My notes had been live only days when I first wanted numbers; a free spot-check showed my domain at zero surface for even my strongest queries — the index hadn't accepted the sitemap yet. So I deferred the paid probe and its weekly cron for three weeks — measuring an un-crawled site produces nothing but noise. Turning the cron off for that window was itself the disciplined move.
An active probe asks one grounded model a controlled question set — clean and repeatable, but a single lens. The complement is the passive report each platform gives about real citations to real users. While my probe was still paused, Bing Webmaster Tools handed me the first hard evidence I'd been cited at all: its AI Performance report logged four Copilot citations on 2026-07-06 — a signal the deferred probe couldn't have caught.
Both lenses matter because of an asymmetry in where each assistant gets its facts: ChatGPT and Copilot lean on the Bing index; Google's AI Overviews and Gemini lean on the Google index. Being cited is partly a question of which index has crawled and ranked you, and no single probe sees all of it.
| Lens | What it sees | Blind spot |
|---|---|---|
| Active probe (Perplexity Sonar) | Whether a controlled question set gets you cited, as a weekly trend you fully control | One model only; only the questions you thought to ask |
| Bing AI Performance report | Real ChatGPT/Copilot citations drawn from the Bing index, tallied passively | Bing-family surfaces only; no control over the prompts |
| Google Search Console | Crawl status and impressions feeding AI Overviews / Gemini | Citation attribution is coarse; no clean "cited" flag |
The probe tells you what happens when the exact question is asked; the reports, what actually happened across a platform's surface. Together they triangulate the truth.
None of this needs more than a hundred lines of Python and a scheduler. The discipline is the product:
owner/repo, App Store ID — never a bare name a stranger could share.chmod 600 env file so the headless run reads it without echoing it.The honest ending: this post is itself an asset I'm about to add to the probe — a new question, a new owned pattern, and a fresh row of zeros I won't panic about for three weeks. GEO advice is cheap and untestable: either measure your citations or stop claiming they're going up. Half a cent a query buys you the right to know.
Q. How can I tell whether AI assistants like Perplexity or ChatGPT actually cite my website?
Fire a fixed set of questions your users would actually ask at a grounded model that returns citations — for example Perplexity Sonar via OpenRouter — then run a deterministic substring check for your domain or owner/repo in the citation URLs and the answer text. Log every run to an append-only ledger so the citation rate becomes a trend you can watch, not a one-off guess.
Q. How much does it cost to monitor AI citations weekly?
About half a cent per query on a grounded model like Perplexity Sonar. A fixed set of a couple dozen questions run once a week is roughly a dime — comfortably under a dollar a month. The cost is low enough that the real constraint is measurement discipline, not budget.
Q. Why should the citation-match pattern use owner/repo paths instead of a bare project name?
Because a bare name collides with same-named projects by other people. A repo called patent-mcp matched someone else's clone and produced false-positive citations that were never mine. Scoping the match to an owned path — your domain, owner/repo, or an App Store ID path — removes the collision and keeps the number honest.
Q. How do I keep API errors and out-of-credit failures from corrupting the citation trend?
Record failed rounds as a distinct error row, such as {"error": "HTTPError"}, and skip those rows in every summary and trend calculation. A 402 out-of-credit run then reads as no data for that week, not as a citation rate of zero — the difference between a gap in the record and a false decline.
Q. When should I start measuring AI citations after publishing?
Not immediately. Citations follow a publish → crawl → rank → cite pipeline, so measuring days after publishing just records zeros that mean "too early," not "not working." Give it weeks, and in the meantime read platform reports like Bing's AI Performance report and Google Search Console — they often show the first citation before an external probe can.