Every Measurement We Have Published Was Taken on One Surface. We Asked the Same Five Questions on a Second One.

Every row in our measurement register was taken on Google’s AI Overview. That is the largest limit on everything we have published, because a finding about how categories differ could be a finding about how one product retrieves. So we took the five buying questions from our matched-category study and asked them again on AI Mode, same market, same three runs, same rule for what counts as a vendor, changing nothing but the surface. The median overlap between the two sets of durable sources is 0.40. Asking the same question two days later gives 0.63. The surface costs more than the day.

One of the five categories shares no durable source at all between the two surfaces, a second returns no durable source on the new surface at all, and the two surfaces disagree about which category is friendliest to vendors.

Disclosure: EchoWi sells AI visibility measurement, and this piece puts a number on the weakest part of our own published work, which is the sort of thing a seller has an interest in leaving vague. The five questions are printed in our measurement register with every domain each surface returned, so the comparison can be repeated against us.

The short version

1. Median overlap between AI Overview and AI Mode is 0.40, across the six cells in two markets where both surfaces answered all three runs.
2. That is worse than two days. The same questions re-asked two days apart on one surface gave a median of 0.63, and two readings the same afternoon give 0.67 to 0.70.
3. One category shares nothing. Ecommerce platforms returns four durable domains on each surface and not one of them is on both lists.
4. The extremes survive and the ordering does not. Web hosting and ecommerce give vendors zero durable slots on both surfaces. But accounting is the most vendor-friendly category on AI Overview and CRM is on AI Mode, so the ranking we published does not transfer.
5. And an empty durable set is a property of one reading, not of the surface. Four cells returned nothing durable on AI Mode the first time. Re-asked, all four came back with one, so a claim we published on that basis is retired below.

Why this was the study to run

Our register holds every measurement behind everything on this blog, and until today every one of those rows carried the same value in the surface column. The limits section of each study says so. What none of them could say is how much it costs.

The matched-category study is the strongest thing we have: five buying questions asked in six markets, with a gradient running from web hosting at 1 vendor slot of 27 up to accounting software at 14 of 37. If that gradient is a property of AI Overview’s retrieval rather than of the categories, the practical advice built on it is advice about one Google product.

So: same five questions, same market, same three runs, same vendor rule, and only the surface changes.

The comparison

The first two columns are vendor-held slots out of durable slots. The last is the overlap between the two sets of durable domains, which is the number that matters, because a share can match while the sources underneath it change completely. Every cell in this table and the next one is the first reading of that question, which matters most for the cells marked not comparable: all of them were re-asked, and what came back is two sections down.

Six cells are comparable. The median overlap is 0.40, the range 0.00 to 0.60.

The second market, and the finding that needs no median

Four comparable cells is thin, so we ran the same design in Spain: the five Spanish buying questions already in the register on AI Overview, re-asked on AI Mode, same day, three runs.

Spain’s own median is 0.29 on two cells, and pooling both markets leaves the headline exactly where it was: 0.40 across six cells. The second market reproduced the first rather than rescuing it.

On the first reading, four of the ten AI Mode cells returned no durable set at all against none on AI Overview, and we published that as the finding that needed no median. We re-asked all four. All four came back with a durable set, so that claim is retired. United States payroll went from zero to twelve, and the twelve are exactly the domains that had been in two runs of three the first time. Spanish payroll went from zero to five, Spanish hosting to two, Spanish ecommerce to two.

So the empty set was a property of one afternoon and not of the surface, and the honest version is narrower: AI Mode reaches the strict bar less reliably than AI Overview does, which is a statement about reading it once. Anyone reporting an empty set from a single reading of this surface is reporting the reading.

What the re-reads did instead is strengthen the number we hedged on. Using the better reading of every cell there are ten comparable cells rather than six, and the pooled median overlap is 0.33, range 0.00 to 0.60. Lower than the 0.40 from six cells, on more evidence, and still far below the 0.63 that two days cost on one surface.

And the sharpest single cell is Spanish accounting. On AI Overview it is the most vendor-friendly cell in the whole study, 5 of 7 durable slots held by companies selling accounting software. On AI Mode the same question on the same day returns five durable domains and not one of them sells accounting: an invoice-data extractor, a software comparison site, a hosting company, an ERP and YouTube. The overlap is 0.09.

What that number means next to the ones we already had

We have two other measurements of the same shape, and this is the first time all three can sit in one table.

One check belongs beside that table, because a Jaccard divides by the union and so reports a small set nested inside a large one as disagreement. Across the 6 cells where both surfaces returned a durable set, the median size ratio is 1.3 and the widest is 2.5, so these are comparable sets and the 0.40 says what it looks like it says.

So the surface is the biggest of the three, and it is the one that every dashboard in this category collapses. A product that reports “your AI visibility” as a single number is averaging across sources that agree less than the same source agrees with itself two days apart.

The category that shares nothing, and the control that makes it a finding

Ecommerce platforms returns four durable domains on AI Overview and four on AI Mode, and the intersection is empty. AI Overview gives Reddit, YouTube, Zapier and a design gallery. AI Mode gives a post-purchase logistics platform, a how-to-build-a-website guide and two development agencies.

That zero is the number in this study most worth doubting, so we re-read the AI Overview side the same afternoon. All four of the register’s domains came back, plus one. The baseline was not stale, and the zero is between surfaces rather than between days.

The ordering does not transfer, and that is a real correction

Our matched-category piece publishes the ranking of categories as the exportable finding, on the grounds that the rate is local and the ordering travelled across six markets. It does not travel across surfaces.

On AI Overview, accounting is the most vendor-friendly category at 6 of 13. On AI Mode, CRM is, at 3 of 9, and accounting drops to 3 of 12. The claim that survives both is narrower and still useful: the categories where vendors get nothing get nothing on both surfaces, and web hosting and ecommerce are zero twice.

The empty set that was not empty an hour later

Thirteen domains were cited across the three AI Mode runs and none of them appeared in all three. That is not the surface failing to answer, which is why it is worth stating carefully: all three runs returned an answer with citations in them, and no source was in every one.

We have seen this exact shape before. In our cross-day work an email marketing question returned nothing durable one afternoon and seven domains an hour later, and those seven were exactly the sources that had been in two runs of three. A strict bar and a shifting retrieval produce an empty set without anything being broken. Reading one question eight times afterwards showed what the shifting actually is: the sources do not flicker, one run in every five to eight answers differently, and the strict bar drops all of them at once when a sample contains it.

Why there is no Gemini column

We ran the same five questions on Gemini too, and the rows are in the register. They do not carry a vendor share, for two reasons that are worth separating.

Three of the five stopped before three runs, so their bar for “cited in every run” is two rather than three and they are not comparable to anything above.

The bigger reason is that we cannot tell a small citation list apart from a small capture. This measurement route is already known to return no sources at all for one major assistant, which means a short list here has two possible causes and we cannot separate them from the outside. Publishing a vendor share on top of that would be publishing an artefact with a percentage sign.

One detail from those rows is worth naming anyway, because it is checkable. The only source that held every Gemini run on the hosting question is a hosting provider’s own blog, on a page comparing itself with its competitors. That is the pattern from our work on who writes the best-tools lists, showing up as the single most durable source on a surface.

The Gemini column, a week later, and why the list was short

The section above says there is no Gemini column because we could not tell a short citation list apart from a short capture. That was honest and it was not the end of it, so we went back with a different call.

It is neither. Asked the same way through the cached single call rather than the repeated one, Gemini returns 9 to 18 citations per answer, which is not a capture problem. It spreads them over 2 to 5 domains.

That is the whole answer. Gemini prints 3.80 citations per distinct domain against AI Overview’s 1.31, medians over six questions in three markets on the same day, and Gemini is higher in 6 of the 6 cells with no overlap between the two ranges. The short list was real and it was not the instrument: Gemini quotes several passages from the same page, so a handful of sources carries an answer that looks well cited.

You can see it in the citations themselves. Gemini’s are text fragments, anchored into a specific sentence on the page. AI Overview’s point at the page.

That gives a third unit, and it is the one a tool gets wrong. Across the six answers Gemini printed 77 citations, which resolve to 23 pages and 22 domains. Its pages are almost exactly one per domain, and its citation count is more than three times either. Anyone counting citation URLs as sources reports a Gemini answer as three times better sourced than it is, and this register has already published a correction for making that mistake in the other direction, counting URLs as pages in our own work.

What that does to a comparison

With a route that captures, the comparison this article could not run becomes possible, and it lands where the rest of the piece would predict. Across those six questions, the two surfaces share 8 domains out of 47.

Two Google properties, the same questions, the same day, and five sixths of what one cites the other does not.

That is the surface finding from the top of this article, measured between two products of a single vendor rather than between vendors. It is the strongest version of it we have, because nobody can attribute this one to a different index or a different crawler: whatever separates AI Overview from Gemini, it is not that they are different companies.

And a caution about the surface this article is built on

While checking which surface cites pages, we found something that reaches back into the number at the top of this piece.

An AI Mode answer prints numbered citation markers in its own body, and the structured list we read does not always carry all of them. Across 5 questions in 3 markets and three languages, AI Overview’s list carried every printed marker in all 5 cells, and AI Mode’s ran from 21 per cent of them to all of them. We first read this off two cells and wrote that it was how each surface gets parsed. A third market returned an AI Mode answer carrying every marker it printed, so it is not that: the capture is intermittent, not halved, and an AI Mode count is therefore a floor of unknown tightness rather than one you can correct by doubling. Gemini, read on 5 of the same questions the same day, also carried every marker it printed. Of the 3 surfaces we can read this way, 2 return their answer’s own citation list intact and the gap is AI Mode’s alone.

Which way that pushes depends on the figure. Where we report AI Mode citing far wider than AI Overview, an under-captured AI Mode makes the real gap larger, so those numbers are floors and the finding is safe.

The overlap median at the top of this article is the one we could not reason our way to, so we measured it. For every cell where we hold the full marker list on both surfaces we computed the Jaccard twice, once from the list the tool returned and once from every marker the answer printed. That is 4 cells now, having been 2 when we first published this paragraph, and the correction we made then survives the widening: 2 cells understate the agreement and 2 overstate it, by a factor of 1.4 to 2.7.

The right-hand column is the whole mechanism, and it is arithmetic rather than a discovery. Restoring a dropped domain that the other surface also cites raises the intersection; restoring one it does not raises the union instead. So the overlap rises exactly when the share of dropped domains the other surface shares is larger than the overlap you already measured, and that holds in all 4 cells because it cannot fail to.

What is measured, and what makes the sign unknowable in practice, is that the share swings from 0 to 3 in 10 with nothing to predict it. Whether your number is too high or too low is decided entirely by the domains the truncation removed, which is precisely the list you do not have. Anyone reporting a cross-surface overlap can compute the correction only by first recovering what was dropped. So 0.40 carries an error of that size in a direction that is knowable in principle and unavailable in fact. That is worse than the floor we claimed here at first, and it is what the measurement says rather than what the mechanism suggested.

One of the 4 cells needs its provenance stated, because the two sides are not symmetric. The Spanish AI Overview answer returned 5 domains and printed no numbered markers at all, so its set comes from the returned list while AI Mode’s comes from the body. It is included because the question here is what AI Mode’s truncation does, and AI Overview supplies only the set to compare against. It is flagged because a reader entitled to check the arithmetic is entitled to know that.

What does not move is the Gemini comparison in this section. Gemini’s list carries every marker its answers print, the same as AI Overview’s, so the 8 of 47 is measured with one instrument on both sides.

And there is a second input we had never varied at all. Every reading in this study is a question, because that is what these products ask; asked instead as the keyword phrase a buyer types, 4 categories of 19 return a substantially different set of sources.

The caution that comes with it

One draw per question per surface, so none of this is a rate. A domain missing from one answer might be present in the next, and this register has measured that a single reading is a draw rather than an estimate.

That matters most for a temptation we had and did not take. Four of the publishers in our crawler study disallow Google-Extended, which is the control over use for Gemini, and all four are absent from these Gemini answers while being cited in AI Overview. It reads like a mechanism confirmed. It is not: only 8 of the 43 non-blocking domains carry over either, so at that base rate the expected number of blockers present was under one and we observed none. Observing zero where you expected zero separates nothing, and the register rows carry that arithmetic next to the result so the number cannot be quoted without it.

What this does not show

2 markets, one day, 10 questions, for the overlap median. This is the United States in English and Spain in Spanish on 12 August 2026, three runs per surface per question. That is the scope of the number at the top of this article and not of the whole piece: the Gemini section and the capture section below were measured later and reach a third market, and each states its own scope where it sits. It is not a time series and nothing here is causal.

6 comparable cells is still a small number, and the better-reading pool of 10 is the same weakness with more rows. The median of six overlaps is a weak statistic, and we would not publish it alone. It is publishable because it sits next to two other medians measured the same way on more rows, because the second market reproduced the first, and because the direction is the same in five of the six cells.

The vendor rule is ours and it is strict. A domain counts as a vendor when its own home page names the product the question asks about. Two pages in this batch, Salesforce and ADP, refuse automated requests entirely; Salesforce keeps the classification it already had in the register from when it could be read, and ADP is marked undecided in its row rather than guessed.

And a surface is not a user. AI Overview and AI Mode are both Google, reached differently. That two Google surfaces disagree this much is the finding; how many people see each is a different question and not one this measures.

Common Questions About Measuring Across AI Surfaces

Does a visibility score on one AI surface predict the others?

Not on this evidence. Asking five identical buying questions on AI Overview and on AI Mode in the same market on the same day gave a median overlap of 0.40 between the sets of sources cited in every run, and one category shared no source at all. That is lower than the 0.63 the same questions gave when re-asked two days apart on a single surface.

Which AI surface should I measure?

The ones your buyers use, measured separately, and the honest answer is that the number of surfaces you can measure is smaller than the number that exist. What this study argues against is a single blended figure: averaging surfaces that agree at 0.40 produces a number that is not about any of them.

Is my category friendlier to vendors on some surfaces than others?

Yes, and the ranking changes. Accounting software was the most vendor-friendly of five categories on AI Overview and CRM was on AI Mode. What held on both was the bottom: web hosting and ecommerce platforms gave vendors zero durable citation slots on either surface.

What does it mean when a surface cites sources but none of them is durable?

It means retrieval moved between runs, not that the surface failed. Our payroll question on AI Mode returned answers with citations all three times, thirteen distinct domains in total, and no domain appeared in all three. A strict “cited in every run” bar reports that as an empty set, which is correct and reads worse than it is.

Why not just report an average citation rate instead?

Because with two or three runs an average is mostly a rescaled version of the same count, and it buys tolerance that only more runs can actually provide. We would rather publish the set of domains, which anyone can check, than a smoother statistic built on the same three readings.

How would I repeat this against you?

Take the five questions from the register, ask each one three times on two surfaces in one market on one day, keep only the domains that appear in every run, and compute the overlap between the two sets. The whole design is in the previous sentence, and the domains we got are printed row by row.