Measurement and maintenance · Branch M.C, AI visibility telemetry
M-08 Build the controlled prompt benchmark around intent classes
A fixed prompt panel must be run periodically with engine, model, date and region metadata, and the discovery, commercial, comparison, local, trust, brand and support classes must be kept balanced.
- Scope
- All public content
- Priority
- High priority.
- Quality bucket
- Measurement and experiment readiness
- Legacy coverage
- GEO-52, GEO-53
Why it matters
If the prompts change from run to run, or if one type of question dominates, the results cannot be compared and the trend means nothing. Generative answers also vary by engine, model, date and region, so results without that metadata cannot be reproduced. A fixed, balanced panel makes changes in visibility something you can actually trust.
How to implement
- Create a fixed prompt set and avoid changing it often.
- Record engine, model, date and region for every run where possible.
- Run the same prompts on a regular schedule, and repeat them several times if needed.
- Record brand_mentioned, recommended, citation, position and sentiment for each response.
- Split the prompts across discovery, commercial, comparison, local, trust, brand and support classes, and keep them balanced.
Checklist
- Create a fixed prompt set.
- Record engine, model, date and region where possible.
- Run the same prompt periodically and, if needed, several times.
- Record fields such as brand_mentioned, recommended, citation, position and sentiment.
- Do not change the prompt set often and break the trend.
- Divide the benchmark into at least the following classes:
- Discovery: "What is X / how do I do it?"
- Commercial: "What should I buy for X?"
- Comparison: "A or B?"
- Local: "Who is in my city or nearby?"
- Trust: "Is the brand reliable / what are the experiences like?"
- Brand: "What does the brand do?"
- Support: "How do I solve problem X?"
How to audit
- Dataset reproducibility.
- Prompt sampling bias.
- Prompt taxonomy distribution.
How UltimateSEO checks this automatically
Partly automated. PASS when a controlled prompt benchmark snapshot exists with at least one intent class group; the full benchmark design is confirmed by the customer. The crawl gives a signal; a person decides the final state with business evidence.
States used: PASS, FAIL, UNKNOWN, NOT_APPLICABLE or BLOCKED. UNKNOWN is never shown as a pass. How states work.
Myth guards that reference this control
Sources
Part of The UltimateSEO Framework v4.1, revised 2026-10-07. Evidence states, control states and the publish gate are explained on the hub.