Your monthly SEO report says rankings are up. Position four to position two on a decent term, green arrows, a nice chart. Meanwhile the phone rings about as often as it did last year. If that's your situation, you're not imagining it and your agency probably isn't lying to you.
They're measuring something that used to be a reliable proxy for business and increasingly isn't.
Ranking and getting found have come apart
For twenty years, position on a results page was a decent stand-in for attention. Rank higher, get more clicks, get more customers. The whole industry's reporting was built on that assumption, and for a long time it held.
Two things broke it.
The results page stopped being a list of links. Ads, a map pack, People Also Ask, featured snippets, video carousels and now an AI Overview that can occupy most of the first screen. You can hold position two and sit below the fold on a phone. Position two isn't what it was in 2018 and it isn't what it was in 2023.
A growing share of searches never reach a results page. People ask ChatGPT, Perplexity or Gemini directly and get a synthesized answer naming a few businesses. If you're not named, you got nothing, and critically, nothing about that appears in your SEO report. There's no impression, no position, no line item. It's invisible loss.
This is the uncomfortable part. A report can be entirely accurate and still describe a shrinking share of how customers find you. Nothing in it is wrong. It's just no longer the whole picture, and it doesn't say so.
Why agencies haven't adapted
Mostly it isn't malice. It's tooling, incentives and inertia.
The tools don't measure it
Rank trackers work by querying a search engine from a set location and recording positions. That's a solved, cheap, automatable problem. Measuring AI visibility means repeatedly asking several models the same questions, parsing prose answers for brand mentions, and accepting that answers vary run to run. It's slower, more expensive and fuzzier. Most agencies' entire reporting stack simply has no field for it.
The contract is written around rankings
If the agreement says "improve rankings for these twenty keywords," then that's the job, and reporting on anything else is unpaid work that risks making the paid work look less impressive. Incentives are doing exactly what you'd expect.
Nobody wants to reopen the conversation
Explaining to a client that the metric everyone has celebrated for three years is losing meaning is a hard meeting. It's much easier to keep sending the green arrows.
Five questions to ask your provider
You don't need to become an expert to get a straight answer. Ask these, in this order, and listen to how they respond rather than just what they say.
1. "For our top ten commercial keywords, how often does an AI Overview appear, and are we in it?"
A good provider will know or will offer to check within a week. "That doesn't really affect us" without any data behind it tells you they haven't looked.
2. "If I ask ChatGPT or Perplexity the questions our customers ask, do we get named?"
This takes ten minutes to test. If they've never run it, that's the answer. If they have, ask to see what was recorded.
3. "Does our robots.txt allow the AI crawlers, and did we decide that deliberately?"
Plenty of sites block them accidentally through a security plugin or a CDN preset. The word you want is "deliberately." An accidental block is a real problem hiding in a config file.
4. "How has our impressions-to-clicks ratio moved over eighteen months?"
If impressions rose while clicks stayed flat or fell, you're being shown and not clicked, which is the fingerprint of zero-click results and AI summaries absorbing the answer. That data is already in your Search Console, free.
5. "What would you change if half our customers stopped using Google within two years?"
This is the real question. You're testing whether they've thought about it at all. A thoughtful "here's what I'd prioritize, and here's what I think is overhyped" is a good sign. Dismissiveness is not.
What a current report should include
Alongside the usual rankings, traffic and conversions, you should be seeing some version of:
- Whether an AI Overview appears for your money keywords, and whether you appear in it.
- Whether the major AI tools name you for the questions your customers actually ask, tracked over time rather than as a one-off.
- Impressions versus clicks as a trend, so zero-click erosion is visible instead of buried.
- Entity health: consistent name, address and phone number, valid schema, profiles connected.
- Where you're mentioned off your own site, since that's what AI weighs when deciding whom to name.
You might not need to fire anyone
Worth saying, because this could read as a pitch and it isn't quite one.
If your provider does solid technical and content work, they're an asset. The foundation they've built is exactly what AI visibility sits on top of, and none of it is wasted. A lot of agencies are perfectly capable of extending into this and simply haven't been asked to. Sending them these five questions is a reasonable first move, and a good one will thank you for the prompt.
Where we'd argue for a change is if you ask and get told it doesn't matter, without data. That's not a tooling gap any more. That's a judgment call about where search is going, and it's one you're paying for.
The thing to avoid is finding out in eighteen months. Visibility in answer engines is built on consistency and third-party corroboration, both of which take months to accumulate. Starting late is genuinely more expensive than starting now, which is not true of most marketing.
Where we sit
We run SEO and answer-engine work as one program rather than two line items, because the underlying fixes overlap almost entirely and splitting them mostly creates an excuse to bill twice. If you want to see where you currently stand across Google and the major AI tools before deciding anything, that's what our audit does, and it's useful whether or not you keep working with whoever you're working with now.