We Asked 19 Categories as a Keyword and as a Question. Four Answered Differently.

Every AI visibility product asks questions, because a question is what someone types into an assistant. But an AI Overview also fires on the keyword phrase a buyer types into Google, and nobody in this category appears to have checked whether the two reach the same sources. We asked 19 categories both ways on the same surface, in the same market, on the same day, then repeated 5 of them in each of three more markets, and finally asked 8 of the American cells again the next morning. That is 26 comparable pairs, and the 11 that came back with identical sources are all in the first one. Across the 3 European markets the number is 0 of 11.

The useful answer is not that form matters, and it is not that form is irrelevant. It is that the answer depends on the market. In the United States the form usually costs nothing and occasionally costs almost everything. In Spain, Germany and France it has cost something in every cell we could compare, and by more than re-reading the same wording costs. Nothing visible about a category tells you which case you are in, and it now looks as though nothing visible about a market does either. And in the eight American cells we came back to the next morning, the day moved the answer further than either form ever did.

Disclosure: EchoWi sells AI visibility measurement, and this study finds a blind spot in how products in this category ask their questions, ours included. Everything needed to repeat it is here: every exact string, the surface, the market, the number of runs, the date, and every domain named in our measurement register. Run it against us and publish a different answer.

The blind spot we found in our own corpus first

This started as an audit of our own register rather than as a study.

Before these rows it held 184 distinct prompts across every study we have published. Two of them were keyword shaped, and both were Spanish and French noun phrases left over from a wording experiment rather than deliberate controls. Everything else was a question, with a verb and usually a question mark.

That is not an oversight peculiar to us. It follows from what these products are for. You buy AI visibility measurement because you want to know what an assistant says about you, and an assistant is a thing you talk to. So the natural unit is a prompt.

The problem is that Google’s AI Overview is not only reached by talking. It fires above ordinary search results, on the phrase somebody types with no verb in it. If the sources behind those two inputs differ, then a product tracking only prompts is reporting on an input a large share of buyers never use.

Nobody had measured it. So the axis is worth a study on its own, and it is a different axis from the wording work already in this register, which varies how a question is phrased while keeping it a question.

The design, frozen before the first call

19 categories, each already measured in our register as a question on AI Overview in the United States in English, so the new readings pool with what we already hold rather than sitting beside it. Each asked in two forms, three runs requested per reading, cache bypassed.

The keyword form is the phrase a buyer types. The question form is the string already in our register.

A source counts as durable when it is cited in every answered run, which is the strict bar the rest of our work uses. answered is read out of the result and never assumed from what was requested, because this surface does not always answer three times when you ask it to.

We wrote the predictions down before any call. If form does not matter, the two sets per category should overlap at roughly the level a plain re-read of the same question gives. If form matters as much as wording does, overlap should fall towards the band where wording variants share almost nothing. And the expensive outcome would be a keyword form that returns no AI Overview at all, which would make this a different and larger article.

What came back

11 of the 15 are not merely similar. They are the same domains, every one, in both directions. Ask an AI Overview “best ai coding assistant” or “what is the best AI coding assistant?” and the ten sources that survive all three runs are identical.

4 are not close to that. Ecommerce shares three domains out of nine seen. Email marketing shares two out of nine. Web hosting and business banking, both added in the second batch, share three of nine and four of twelve.

Four pairs are excluded, and that is worth stating rather than hiding. Cloud storage answered twice on one side and three times on the other, and unanimity over two runs is a different bar from unanimity over three, so comparing them would be comparing two rulers. Three more failed on the question side: project management and headphones each answered once against three, and fractional CFO services did not answer at all while its keyword form answered twice. A single answered run is a draw, not a durable set. All four pairs are in the register with their real counts.

And that is a second observation we are deliberately not calling a finding. In three of the four excluded pairs it was the question form that answered fewer times, and in one it was the keyword. Three against one is nothing. It is recorded so a larger sweep can test it, not because we believe it yet.

The control, which is the actual finding

A cross-form overlap of 0.22 looks conclusive and proves nothing on its own, and this is the part most studies of this shape skip.

Both categories that diverged are categories our register already knows are unstable. An unstable category disagrees with itself when you read it twice. So a low overlap between a keyword and a question could be the form, or it could be that this category would have given you two different answers to the same question anyway.

There is exactly one way to separate those, and it is cheap: read the same form twice and see what it costs.

Email marketing is clean. Asked the same question twice, it returns the identical six domains. Asked as a keyword instead, it shares two of them. There is nothing unstable happening in that category on that day; the form is doing all of the work.

Ecommerce is mixed and the honest reading says so. Its own question re-read shares 0.60, well short of identical, so part of that 0.33 is a category that moves around on its own. But 0.33 is lower than 0.60, so the form is costing something on top.

Without those two extra readings the table above would have been a finding about form that was partly a finding about instability, and no reader could have told.

And then we re-read the other side of both, because the rule this study produced applies to this study. A cell that yields the result you are hunting gets re-read before it is counted, and until now only the question forms had been. Ecommerce’s keyword form returned the identical 8 domains, so its 0.33 rests on two stable readings. Email marketing’s keyword form answered three times rather than two on the re-read, which is a different bar and not comparable, but the two domains that survived it are in neither of the question form’s readings. Both divergences hold.

What the control is worth, in cells

The second widening is the first time we can put a number on the paragraph above, because 7 cells across the study have now been read twice in the same form. 3 of the 7 stopped diverging once they were.

Password managers looked like a clear divergence: the question form returned 4 durable domains against the keyword form’s 9, an overlap of 0.44. Asked the same question again the same day, it returned exactly the 9 the keyword form had. Website builders did the same thing, 3 against 8 on the first reading and the identical 8 on the re-read. The third is the scheduling cell below, where the same mechanism took a durable set all the way to zero.

Without those extra calls this study would be reporting 7 divergent categories out of 15 rather than 4, and the sentence about the form usually costing nothing would have been retracted. It was written down before the calls that three or more new divergences would force that retraction, and on the first readings there were four.

Two things have to be said about that number so nobody quotes it as a rate.

• The selection is not random and we are not pretending otherwise. A cell gets re-read because its first reading looked divergent, so “7 of 7 looked divergent” is the protocol describing itself, not a finding. The only figure that carries information is the 3 that resolved.
• The re-read does not reliably return more. Password managers went from 4 durable domains to 9 and website builders from 3 to 8, but web hosting went from 6 down to 4 and business banking returned the identical 9 both times. Whatever is moving is not a warm-up effect.

The uncomfortable version, for anyone who sells a number like this: a single reading of a question-form prompt can understate its own durable set badly enough to invent a difference that is not there, and nothing inside that reading says so. The tell is not in the statistic. It is that you asked twice.

The cell where the statistic said zero and the sources were the same

Scheduling is worth its own section, because it nearly entered the table as the most extreme disagreement in the study and it is not a disagreement at all.

The keyword form answered all three runs and cited ten domains. Not one of them appeared in all three, so its durable set was empty and the overlap against the question form was zero. The question form, same surface and day, returned nine domains in every run.

Zero against nine is as far apart as this measure goes. But the nine domains the question form returned are exactly the nine that the keyword form had in two runs of three. One run had dropped all nine at once and added google.com instead. Re-read, the keyword form returned the same nine in all three runs.

So the sources were identical the whole time and the statistic said total disagreement. That is a property of a unanimity bar, which this register has measured before in other studies: a single degenerate run empties the numerator, and with three runs there is no tolerance to absorb it.

Two things follow, and the second is the uncomfortable one:

• This cell counts as identical, on the better of its two keyword readings, and that choice is declared rather than buried. Counting the zero would have been counting the bar rather than the sources.
• The measure this whole study uses can manufacture the exact result the study is looking for. We found it here because a zero next to a nine was too strange to accept, and the honest reading is that a smaller gap produced the same way would have gone straight into the table.

What this means if you are buying or building measurement

A prompt is not a proxy for a keyword, sometimes, and where you sell decides how often. Everything in this section is the American arm. If your category is one of the 11 that answered identically, you can track prompts and sleep well. If it is one of the 4 that did not, your prompt-based report is describing a set of sources that the keyword-driven half of your buyers never sees.

You cannot tell which you are from the outside. There is no property of these categories visible before measuring that predicts the split, and the second batch was chosen to test one. The four divergent categories are ecommerce platforms, email marketing, web hosting and business banking, so the obvious guess is that they are the shopping-adjacent ones. It does not survive the identical column: VPN services, password managers and website builders are bought the same way and answered identically both ways, and business banking is not shopping-adjacent at all. Category, market and vertical do not sort them.

So the check is the cheap one: ask it both ways once. You do not need a programme. Take your five most important questions, ask each as the keyword a buyer would actually type, and compare the sources. If they match, stop and go back to tracking prompts. If they do not, you have just found out that half your measurement is aimed at the wrong input.

Before any of that, check the day. In the eight American cells we re-read the next morning, the same wording moved more overnight than the two forms moved against each other. If you are comparing this week’s report with last week’s, the first thing to rule out is the calendar, and the way to rule it out is to ask the same string twice on the same day and see how far apart those two land.

And ask your vendor which one it sends. This is a fourth question to put to a supplier, after the three this corpus already argues for: whether their runs bypass the cache, how many wordings they follow per intent, and what happens to their stability figure when the number of runs goes up. Add: do you ever send the keyword form, and do you report it separately.

The same question on a second surface

Everything above is Google’s AI Overview, and this corpus has measured that a result on one surface does not travel to another: the median durable-source overlap between AI Overview and AI Mode is 0.40, worse than asking the same question two days apart on one surface.

So the question worth asking was not whether the same categories diverge on AI Mode. It was whether the form effect exists there at all. We took 4 categories, the two that diverged and two that were identical, and asked the same strings on AI Mode.

The form effect reproduces. Both categories that were identical on AI Overview are identical on AI Mode, and email marketing diverges again and harder: 8 durable domains from the keyword form, 5 from the question form, and only YouTube in common.

Ecommerce could not be measured and that is stated rather than counted. Its question form returned no durable set at all on AI Mode, in two separate readings, once across 45 domains and once across 32 of which every single one appeared in exactly one run. An empty set forces the overlap to zero whatever the other side holds, so calling that the study’s largest divergence would have been reading the bar rather than the sources.

And there is a separate observation about the surface itself. 4 of the 11 AI Mode readings returned no durable set, three of them with the same signature: a row of domains sitting at exactly two runs of three, which is one run diverging completely and emptying a unanimity bar. Re-read, two of the three came back fully. Anyone measuring AI Mode at three runs against a strict bar should expect that, and should expect it to look like a finding.

The same design in a second market, and nothing survives the trip

This piece has been saying for two widenings that what would move it is a market or a surface other than this one. The surface arm is above. Spain is the market arm.

Five categories, each already a cell in the United States arm of this same study, so the comparison is category against category rather than Spain against a different question list. Two of the five were identical in the US and three diverged.

Not one of the five reproduces its United States result. 4 of the 4 cells that can be measured diverge, including the one that was identical both ways in English, and in all 4 the cross-form overlap sits below that cell’s own same-form control. The form is doing work on top of instability in every one of them.

The fifth cell is the sharpest result this study has produced. Asked as a question, ¿Cuál es el mejor gestor de contraseñas? returns an AI Overview. Asked as the keyword a buyer types, mejor gestor de contraseñas returns none at all: 2 readings, 6 runs, zero answers and zero errors. That is not a low overlap and it is not an instrument failure. It is the surface declining to summarise, and it means a Spanish buyer typing the keyword has no AI answer to be visible in, while the same buyer asking the question does.

It is also the outcome the original design named as the expensive one, written down before any call and reached on the fifth category of the third batch.

This arm needed its own comparison bar, and that is a finding rather than bookkeeping. 5 of its 19 readings stopped before the three runs they asked for, against almost none in the United States batches. So a Spanish cell is compared only where both forms have a reading that answered three times, and within those the larger durable set counts. The United States figures above are untouched by that rule and are still computed the way they always were.

What this does not do is make the United States finding wrong. It makes it United States English: 11 of 15 identical there, 0 of 4 identical here, same surface, same method, same bar.

The third market, and the United States turns out to be the outlier

Two arms disagreed about this study’s own headline. The United States said the form usually costs nothing, 11 of 15. Spain said it always costs something, 0 of 4, and by more than a same-form re-read costs. Four cells cannot tell you which of those is the general case, so we ran a third market with the design and the prediction written down first.

Germany lands with Spain.

4 comparable cells, 0 identical, and all 4 below that cell’s own control. The right-hand column is the only thing that makes the left-hand one mean anything: it is what the same wording costs when you simply ask it again, so a cross-form number below it is form doing something that re-reading does not.

Business banking is in the register and is not in that table. Its question form answered three runs and both keyword readings stopped at two, and a two-run durable set is a different bar on a unanimity measure. Its two keyword readings do compare with each other, at 57 against a comparable pair of 4.

This arm changed one thing about the method, and it matters more than the result. The earlier arms re-read a cell because it looked divergent, so “7 of 7 controlled” describes the procedure rather than the study, and the register carries a separate key saying so. Here every cell got its control whether or not it looked divergent. That costs five extra readings and buys two things: a control number a reader can take at face value, and the first look this study has ever had at what a control does on a cell that agreed.

2 of the 5 came back identical, 100 and 100. So a same-form re-read can return exactly the same durable set, which is what makes the cross-form 20 and 50 in those same categories hard to explain as noise.

The caveat on those hundreds, because it is the one we would want a reader to raise: both were cells where every domain the keyword form cited survived every run, so there was no tail for the statistic to disagree about. This register already calls that a low bar rather than a fixed set, and hosting shows why: its first keyword reading had 9 of 9 durable and its control had 5 of 9.

The fourth market, and the cell that nearly forced a retraction

Three markets is still thin for a word as large as Europe, so we ran a fourth: the same five categories, both forms, three runs each on AI Overview in France in French, on 16 August 2026.

France lands with Spain and Germany. 3 comparable cells, 0 identical.

The two categories missing from that table are the more interesting half.

Password managers returned no AI Overview at all for the keyword phrase. 0 answered runs across 6, in two separate readings, while the question form of the same intent rendered and cited 5 stable domains. That is not a different source set, it is a different fact: in that cell the form decides whether there is anything to be cited in at all. It is the same distinction our local study is built on, arriving from a direction we were not looking in. Web hosting dropped out for a duller reason: its keyword form never held three answered runs across two readings.

Business banking is the cell with a control, and the control holds. Its keyword form was read twice at three runs and returned the same six domains both times. So the 71 against the question form is form doing something, not a reading disagreeing with itself: the keyword form keeps a personal-finance comparison site the question form drops, and the question form keeps an accountancy service the keyword form drops.

The ecommerce control does not hold, and we are counting that cell as divergent anyway, which is worth arguing with. Its question form read twice shares 3 of 5. A cell whose own form disagrees with itself that much cannot carry a claim about what form does. It is in the 0 of 3 because the count is of cells that were not identical, which it was not, and it is named here so nobody quotes it as evidence about form.

The reading that would have made us retract a correct sentence

The CRM cell was read first at two answered runs on each side, because the surface stopped early both times. Both forms returned the same six domains. Identical. That is Europe’s first identical pair, it would have replaced “0 in Europe” with a count in four languages, and it is wrong.

Read again an hour later at three answered runs on both sides, the durable sets are 7 and 6 sharing 5. The keyword form gains two domains that survive three runs, the question form keeps one the keyword form drops below the bar.

The mechanism is not subtle and this register already had it written down: a unanimity bar over fewer runs is a lower bar, so shallower readings agree more. What the cell adds is the direction of the damage. A thin reading did not produce a cautious result here, it produced a retraction, and a retraction feels like the humble direction to move in. It is not automatically the safe one, and this is the first time in this corpus that being underpowered pushed toward publishing a correction rather than away from one.

Which is also why every comparable cell in every arm of this study requires three answered runs on both sides. The rule was written for the German arm, and France is where it paid.

One instrument note, and the second half of it has been withdrawn. 8 of 17 French readings did not deliver three answered runs, against 5 of 18 in the German arm. That count stays, because it is why France has 3 comparable cells and not 5. What we wrote next was that whether an AI Overview renders at all appears to vary by market, and that does not survive being measured properly.

Those readings were re-read when they came back short, so the count above is partly our own retry policy rather than the surface. Asked cleanly, one reading per cell and no retries in any of the four markets, the overview rendered on every run in 37 of 40 cells, Germany answered every requested run, and every shortfall in the study was the same single category. Render failure is a property of the question far more than of the market, and the sentence that said otherwise is corrected here rather than quietly dropped.

What that does to the headline

Across four markets there are 26 comparable pairs and 11 identical ones, and all 11 are in the United States. Put the three European markets together and it is 0 of 11.

We are not going to dress that up as a rule about Europe. Three markets is three, and the mechanism is not measured: it could be language, it could be how much comparison content exists in each market, it could be something about how the surface behaves outside English. What we can say is narrower and enough to change what a buyer does: the finding that form usually costs nothing was measured in one market and has not reproduced in either of the other two we tried.

The control numbers are not comparable across the three arms, and that is our fault

Every same-form pair in this study where both readings answered three runs, on AI Overview:

Do not read that middle column as three markets. All three arms re-read a cell because it had already looked divergent, but what decides whether that matters is not the rule, it is whether the rule left anything out. In Spain all four comparable cells were divergent, so the rule selected nothing away. In the United States it covered 7 of 15, and a number drawn only from the cells we suspected describes our suspicion rather than the market.

We are publishing the table anyway, with the third column attached, because the alternative is publishing three medians and letting a reader assume they mean the same thing. Two of them can be read as written and the American one cannot.

That is what the design change bought. In Germany the comparison this whole study rests on can be made cleanly: an unselected control median of 79, and four cross-form values of 40, 20, 17 and 50, every one of them below it. Spain gets there by accident rather than by design, because its rule happened to catch everything. In the United States the comparison is available cell by cell, which is how it is reported above, and not in aggregate.

An earlier version of this section put the price of fixing that at five readings and said two arms were affected. Both were typed rather than counted, and counting them gives 8 uncontrolled cells in the United States and none in Spain. The corrected version is the table above.

France is deliberately not in that table. Its same-form re-reads are reported in its own section instead, because two of them compare readings taken at different depths and one of them is the depth comparison itself. Dropping them into a column headed “median overlap” next to three arms measured at a fixed three runs would be the exact mistake this section exists to name.

We went back a day later, and the day moved more than the form ever did

This section exists because of a repair that failed. The American control median sits on the cells we re-read because they looked divergent, so we went to re-read the 8 that nobody had suspected. Every American reading in this study is from 15 August and those eight are from the 16th, which makes them the same wording a day apart rather than the same wording minutes apart. They cannot stand in for a same-day control, and a same-day control cannot be added later.

They answer a better question instead.

All 8 of those cells are cross-form identical. Two different wordings, asked on the same day, returned exactly the same durable set in every one of them. Asked again a day later with the identical keyword string, the median overlap is 41, from 17 to 100.

Six of the eight move more overnight than they moved between a keyword and a question. In those six the thing we were measuring is smaller than the thing we were holding constant.

What that does and does not mean. It does not overturn the study: within a day, in these eight American cells, the form genuinely cost nothing, and that is what the study claims. It does put a size on the claim. If you are measuring your own visibility and you compare a keyword report from Monday with a question report from Tuesday, the difference you see is more likely to be Monday against Tuesday than keyword against question.

It also says something about buying measurement. A tool that runs your prompts once a day and charts the difference is charting this, and eight cells at a median of 41 is a wide chart to read as movement.

Eight cells in one market on one pair of days. The register already carries a separate two-day study on different questions with a median around 62, so this 41 is not a rate and we are not treating it as one. What it is, is the first time this study has had the day and the form in the same table.

And the same eight cells in the question form: the form does not change the drift

The obvious follow-up, written down before it was run: if the form is irrelevant within a day, is it also irrelevant across days? Every product in this category runs questions, so if questions drift more than keywords, the industry’s default is the noisier one.

They do not. The same 8 cells, question form, re-read against their own 15 August readings: keyword steadier in 2 cells, question steadier in 0, and 6 exact ties. Medians 41 and 35, and the gap between those two comes entirely from the two cells that were not ties. The threshold set before measuring asked for 7 or 8 of 8 in one direction before naming one, so this is a null and we are reporting it as one.

The six ties are exact, and that is the part we did not go looking for. They are exact because both forms returned the same durable set on both days, so their overnight overlaps are the same number. Which means the two forms still agreed on the 16th in six of the eight cells, and disagreed in two: CRM at 56 and payroll at 28.

So the finding this article leads with, that within a day the form usually costs nothing, was itself measured on one day, and on the next day it cost something in a quarter of these cells. That is not a retraction: the American arm’s headline stands on 15 August and the German arm’s on the 16th. It is a size for how firmly to hold it.

This second result was not pre-registered. It came from noticing that six numbers matched exactly, which is the kind of thing you find after the fact, and we would rather label it than let it read like a prediction that came true.

Limits

Two surfaces, unevenly. All 19 categories on Google’s AI Overview and 4 of them repeated on AI Mode. It says nothing about Gemini or ChatGPT, and our own work has repeatedly found that a result measured on one surface does not travel to another.

4 markets, 4 languages, 2 days. The United States in English on 15 August 2026, and Spain, Germany and France in their own languages on 16 August 2026, plus 8 American cells re-read on the 16th which are reported only in the section on the day. Every figure above says which market it belongs to. The only figures pooled across markets are the four-arm counts in the section on the headline, and they are labelled as such.

15 comparable pairs. That is enough to establish that both outcomes exist and not enough to estimate how often each occurs. Treat “11 of 15” as an illustration of a split, not as a rate.

The strict bar is strict. A source cited in two runs of three does not count as durable, so a single unlucky run can drop a domain out of a set entirely. That is a known property of this measure and it is why the controls matter more than the headline.

Nothing here is causal. No page was changed and nothing was measured before and after. This is what the two forms returned, not evidence about why.

Common Questions About Keyword and Prompt Form

Is a keyword search and an AI prompt really the same query?

They are not the same string and this study is about whether that matters. In 11 of 15 comparable categories the sources behind them were identical, so for those the distinction is academic. In 4 they were nearly disjoint, so for those it is the difference between measuring your visibility and measuring somebody else’s.

Why does a difference in sources matter if the answer names the same products?

Because the sources are what you can act on. You cannot easily change which products an assistant recommends, but you can get published, cited or reviewed on a site it draws from. If two forms of the same question draw from different sites, the list of places worth appearing in changes with the form.

Which categories were identical and which were not?

CRM, AI coding assistants, running shoes, flight booking, grammar checkers, payroll, scheduling, password managers, VPN services, note taking apps and website builders returned the same durable sources both ways. Ecommerce platforms, email marketing, web hosting and business banking did not. We can see no property of those categories that predicts the split, which is the uncomfortable part.

How do I run this check myself?

Take a question you already track. Write the keyword phrase a buyer would type for the same intent. Ask each several times on the same surface and the same day, keep only the sources that appear every time, and compare the two sets. Then read the same form twice, because without that control you cannot tell a difference caused by the form from a category that simply moves.

Does this mean prompt-based tools are wrong?

No. It means they are incomplete in a way nobody has quantified, including us, and that the size of the gap is a property of your category rather than of the tool. A tool that asks only prompts is right about prompts.

What would change your mind about this?

More categories, and we have now run two widenings. This piece first published five comparable pairs with three identical; six more took it to nine with seven identical; six more again took it to 15 with 11 identical. The proportion has barely moved across the three, which is the least dramatic outcome available and the reason we still refuse to call it a rate. What would move it is a market or a surface other than this one, because everything here is US English on AI Overview.

Where this leaves you

The finding is small, cheap to reproduce, and awkward for everyone selling in this category, us included. 4 of 19 categories answer a buyer’s keyword with a substantially different set of sources than they answer the equivalent question, and two of those four do it while being perfectly stable when asked the same way twice.

That is not an argument for abandoning prompts. It is an argument for spending one afternoon finding out whether your category is one of the ones where it matters, because the check costs almost nothing and the alternative is assuming an answer that is wrong about 4 categories in 19.