AI 可见性并非骗局，但多数衡量方式经不起数据质量检验

If an AI visibility dashboard helped you win budget this year, read four datasets before your next quarterly review. Together, they show AI assistants send a sliver of the traffic Google sends. They also show that none of these datasets could isolate the hidden AI influence many of us assumed was turning up in Direct and branded search.

That doesn’t make AI visibility a sham. It means most of the ways we measure it wouldn’t pass a basic data quality test.

One Event, Or The Start Of A Trend?

On Oct. 22, 2026, Rand Fishkin published a 5-Minute Whiteboard on the SparkToro Blog titled “Is AI Visibility a Marketing Boondoggle?” The next day, Amanda Natividad, SparkToro’s chief evangelist, sent her Audience Research Newsletter asking, “What if AI visibility is a sham?”

A long time ago, I learned to separate events from trends. Two warnings from co-authors of the same book, Zero Click Marketing, who work at the same company and publish a day apart, look more like one event than two.

But the data underneath their warnings came from other people. And it has been piling up.

The Traffic Numbers Are Small, And The Conversion Story Is Mixed

Marie Haynes looked at 12 months of GA4 data across 100 websites. ChatGPT visitors converted at 13.8%, against 3.02% for Google organic. That’s more than 4.5 times better, and it’s the stat AI visibility vendors will quote.

But here’s the stat they won’t. Google still drove 96.5% of total conversions in her dataset, and Google organic drove 27 times more conversions than all LLMs combined. She also found that the exponential growth in ChatGPT sessions ended in August 2025, around the release of GPT-5.

SEO Works found a narrower edge. AI referrals converted at 7.18% versus 6.08% for organic, a pooled ratio of 1.18. Yet AI traffic converted worse on 23 of the 63 sites with enough volume to compare. For B2B product businesses, AI converted at 0.62 times the organic rate.

The line from that study I’d tape to every SEO manager’s monitor is this one. A 5% lift in organic search delivered 10.3 additional conversions per site per month. Doubling AI referral traffic delivered 6.6.

The Hidden Influence Hasn’t Shown Up Yet

The usual rebuttal is that clicks were never the point. People see a brand in a ChatGPT answer, don’t click, and then search for it or type in the URL later.

Kevin Indig went looking for that. At first, eight of his 12 properties showed Direct traffic growing from August to September. Then he filtered for engaged sessions. Across the filtered January-September sample, total Direct rose 140.9%, while engaged Direct changed by −1.0%. On one site, GA4 recorded 409,000 Direct sessions, and only 198 were engaged.

That’s bots, not buyers.

He then ran 632 tests on four brands, comparing weekly AI visibility with Direct traffic and branded search at delays of zero to six weeks. When he shifted the data to deliberately wrong dates as a placebo check, several of those mismatched alignments correlated as strongly as the real ones. His conclusion: “An effect could still exist. I just couldn’t isolate it.”

Natividad tied this back to an Ahrefs study of 75,000 brands from May 2025. Branded web mentions correlated at 0.664 with AI Overview brand visibility, and branded anchors at 0.527. Branded search volume came in last at 0.392. If AI visibility reliably produced branded search, she argued, that number should be much higher.

See also: Direct Traffic & Popularity – Correlation, Not Causation

What Fishkin And Natividad Actually Concluded

Neither of them said AI doesn’t matter. Fishkin wrote that AI is “probably quite influential on some buyers in some sectors.” He credited Mike King with the better frame, that AI is “more like a brand channel, a digital billboard if you will, and it should be invested-in and measured as such.”

Natividad put it more sharply. “Familiarity itself is a marketing outcome.” Then she added the sentence that matters most for anyone building a dashboard. “If we can’t prove it, we shouldn’t trust the attribution just because it seems right.”

The Measurement Industry Saw This Coming In May

The SEO world is arriving late to a conversation the PR measurement world started months ago.

On May 20, 2026, AMEC, the International Association for the Measurement and Evaluation of Communication, launched its GEO Principles at its Global Summit in Dublin. Two days later, I admitted that I’d been treating AI citations as the new search rankings and that framing was wrong. Appearing in an AI answer is an output, whether it changes what people understand, believe, or do is an outcome.

AMEC has also gone a layer deeper with a companion white paper, “AMEC Data Quality Principles: Tackling the Unstructured Data Quality Challenges in the AI Era.” Its lead author is Rob Key, founder and CEO of Converseon, and it was produced with the Global Data Quality Initiative, a coalition that includes the Insights Association, ESOMAR, and the Market Research Society.

The paper was written for social listening and media analysts, not SEOs. Read it with an AI visibility dashboard open in another tab anyway. It proposes four questions any data set should pass before anyone acts on it.

The third question is the one that should keep SEO directors up at night. AMEC calls it predictive and construct validity. A metric can be “internally consistent but externally untrue,” the paper warns, and in an AI era that kind of data “does not stay quarantined; it propagates.”

That is a precise description of most AI visibility scores. They’re consistent from week to week. They go up when you publish more. Whether they predict anything a business cares about is, after Indig’s 632 tests, still unproven.

Two more lines from the paper belong in every GEO pitch deck. Principle four says, “Evaluations performed once and stored on a shelf are decorative, not protective.” And its crosswalk with AMEC’s GEO framework says outputs are directional and should be tested “across tools, prompts, and time.” A single screenshot of ChatGPT recommending your brand is an anecdote.

Principle five goes further. It tells practitioners to avoid manipulative GEO, meaning content engineered to game generative AI rather than inform it. AMEC’s phrase for it is “promotion over precision.”

One caution in the spirit of the paper itself. Some of its most striking figures, including the claim that off-the-shelf sentiment models hover at 60% to 65% precision, come from a Converseon-sponsored benchmark, and Converseon sells the cure. The paper calls itself non-promotional, and I think its principles stand on their own. But apply its Source question to the paper, too.

Where I Come Down

I don’t think AI visibility is a boondoggle. I think the AI visibility score is.

The datasets above don’t prove AI has no influence. They prove we’ve been grading a brand channel with traffic-channel math, then filling the gap with scores nobody has validated against behavior. That’s how a reasonable idea turns into a budget line you can’t defend.

Mike King’s billboard frame is the right one, and it fits what I’ve been arguing in my columns on YouGov’s brand research. AI brands are winning on consideration before they win on trust. If AI answers work like a billboard for your brand, measure them the way brand marketers have measured billboards for decades, with exposure, recall, and consideration, and stop asking them to behave like a search result.

How To Measure AI Visibility So It Survives A CFO

Demote AI referral traffic to a diagnostic. Keep reporting it. Stop setting targets on it. Then run the SEO Works math on your own GA4 property. Calculate what a 5% lift in organic conversions is worth next to doubling your AI referrals. If organic wins, as it did across their sample, your roadmap should say so.

Run your dashboard through AMEC’s four questions. Before an AI visibility number goes into a deck, document which tools and prompts produced it, whether results were logged in or logged out, and how bot sessions were filtered. Define whether you’re counting a mention, a citation, or a recommendation, because they aren’t interchangeable. If you can’t answer those, the number fails AMEC’s Source and Accuracy questions before anyone asks about results.

Test your score for real-world validity, Indig style. Pull weekly AI visibility for your brand and line it up against engaged Direct sessions, branded search, and pipeline at lags of one to six weeks. Then shift the series to wrong dates and run it again. If the wrong dates correlate as well as the right ones, you’ve learned your score isn’t predicting anything yet. That’s worth knowing before your vendor renewal.

Build a governed prompt library and re-run it on a schedule. Pick prompts that map to real buyer questions, not prompts that flatter your brand. Run them across several AI tools, save the outputs, and repeat monthly or quarterly. AMEC’s principle is that accuracy isn’t a point in time, and the same holds for visibility.

Ask people, then triangulate. Indig’s recommendation is self-reported attribution. Add a free-text “How did you hear about us?” field to demo and signup forms, and add AI assistants to your brand lift surveys. Natividad warns that this data has its own flaws, so treat it as one input. Her framework of exposure, behavior, and business outcomes is a sensible way to weigh them together.

Put money where the evidence overlaps. SEO Works found the sites gaining AI referrals were the same sites improving in organic, with a correlation of +0.557. Ahrefs found branded web mentions tracked AI Overview visibility better than branded search did. Earned authority does double duty. That’s the least glamorous budget line on your plan and, right now, the best supported one.

The Bottom Line

Fishkin asked whether AI visibility is a marketing boondoggle. My answer is that it becomes one the moment we report a number we haven’t checked against what customers actually do.

More Resources:

• Brands Are Tracking AI Visibility, But Are They Measuring The Right Things?
• Why AI Visibility Does Not Only Depend On SEO
• AI Visibility Rankings Aren’t Stable – New Research Shows It’s Mostly Statistical Noise

Featured Image: PeopleImages/Shutterstock