Recall@k
relevant passages in top k / relevant passages
Whether your passage is among the top k retrieved for each query.
Validation · Retrieval Validation & Testing
Measured before. Measured after.
Every change we make is meant to move a number. Validation re-runs the query set against your pages and your competitors', using the same measures as the original diagnosis, and reports what changed.
The retrieval problem
Without a re-test, nobody knows what the work achieved.
Visibility work is usually reported with screenshots and anecdotes. Retrieval can be tested: the same query set, the same corpus, the same scoring, before and after. If a score did not move, that is worth knowing too.
What we measure
The numbers this work is built on.
Each is calculated for your pages and your competitors', against the same buyer queries, before anything is changed.
Mean reciprocal rank
MRR = (1/|Q|) · Σ 1/rankᵢ
How high your best passage ranks, averaged across the query set.
Score movement
Δ = score after − score before
The change in every measure from the original diagnosis, query by query.
Answer share
k / n
In how many of n sampled AI answers you are named, before and after.
The work
What we do.
Fixed test set
Fix the query set and competitor corpus at the start, so that before and after are comparable.
Retrieval tests
Run each query against an index of your pages and your competitors' and record where you rank.
Answer sampling
Sample AI answers repeatedly, because a single answer varies from run to run.
Technical checks
Confirm every changed page is crawlable, indexable, rendered and marked up as intended.
Deliverables
What you receive.
Validation report
Every measure, before and after, per query and in total.
Technical retrieval check
Crawl, index, render and markup checks for every page changed.
Next priorities
Where scores did not move enough, and why.
How it starts
From application to roadmap.
-
01
Apply
Answer six short questions about your business. We read every application.
-
02
15-minute consultation
Free. We show you two or three verified retrieval weaknesses in your market.
-
03
Retrieval Opportunity Report
A measured diagnosis of one service category, and a prioritised roadmap.
-
04
Implementation
The roadmap carried out, then re-tested against the same measures.
Common questions
Why sample AI answers more than once?
AI answers vary between runs. A single answer can name you, or miss you, by chance. Repeated sampling gives a proportion that can be compared over time.
Can you guarantee the scores will go up?
No. We guarantee they are measured the same way both times, so you see exactly what changed.
The rest of the roadmap
Free 15-minute consultation
Find out where your business stands.
- How search and AI systems currently interpret your business
- Where competitors are better understood than you
- Specific retrieval weaknesses, checked before we speak
- Whether there is work worth doing, and what it is worth to you
Apply for a free 15-minute consultation
We reply within one business day, and usually within the hour.