Reporting: An Honest Summary for a Stakeholder

In this lesson

Measurement is partly there so you can explain it to someone else: a manager, your team or a client. This lesson covers how to write an honest weekly or monthly visibility summary.

What stakeholders ask

Stakeholders usually want three things:

1. Where do we stand?
2. What changed since last period?
3. What are we doing about it?

An honest summary answers all three, and says how far each answer can be trusted.

What an honest summary contains

1. Period and date: which week or weeks?
2. Scope: which brand, country, language and, where set, place or channel. How many tracked questions, and which competitor set?
3. Engine health: how many engines were measured healthily? If a week was partial, say so.
4. Headline number and comparability: SOV and the change against the previous week. If the change is hidden, give the reason.
5. Unmeasured states, kept separate: "no AI answer", "refused" and "not measured" sit on their own line, and none of them is written as 0%.
6. What we did and what changed: work done this period, plus any version changes (a competitor added, a question reworded).
7. What we're not claiming: that the measurement is a sample, and that a change can't be pinned on one piece of work.
8. Next step: one or two concrete tasks.

Three traps

1. Not measured isn't zero. Before writing "0% on Gemini this week", check whether Gemini was measured healthily that week. If it wasn't, the right wording is "Gemini couldn't be measured healthily this week". As Module 2 showed, 0% is a result, while an unmeasured week is the absence of one. If no brand in the set was named (0/0), SOV isn't measured either, and that isn't 0%.

2. One query misleads. A stakeholder may send a screenshot saying "the AI never mentions us", or "we come up first". Either way it's one sample: one person, one moment, one engine, one account. Answer with the measurement: "That's a single answer. Across the full set of questions on five engines this week, here's where we stand."

3. Change isn't cause. "We launched the Q&A page and SOV rose 5 points" claims cause and effect. Better: "We launched the page at the end of week 1. Weeks 1 and 2 are comparable, and SOV rose from 30% to 35%, 5 points. The measurement doesn't show the page was the only reason."

Two summaries (representative)

Acme Helpdesk, its competitors and the numbers are fictional.

Weak summary:

"Our AI visibility dropped from 35% to 26%. We're at 0% on Gemini. We asked ChatGPT last week and came up first. The Q&A page worked."

It was written for week 4 and makes four mistakes. It reports a drop that should be hidden: week 4 is partial, and Competitor C was added in week 3. It counts Gemini, which wasn't measured, as 0%; the accurate thing to report for Gemini that week is its measurement coverage, not an SOV. It presents one query as evidence. And it claims cause and effect.

Honest summary:

Period: week 4. Scope: Acme Helpdesk · United Kingdom · English · online; 20 tracked questions; competitor set A, B, C. Engine health: 4/5. Fewer than 95% of Gemini's scheduled questions were measured, so Gemini couldn't be measured healthily this week; it was left out of SOV, and that isn't 0%. SOV: 26% · partial · 4/5 engines. No change is shown: this week is partial, and Competitor C was added in week 3. What we did: corrected the category, price and trial length on our review-platform listing. What we're not claiming: this is a sample of the week's answers. To judge the effect, we're waiting for at least two comparable full weeks on the new competitor set. Next step: update the 2024 blog post that still shows the old price.

It's longer, but every sentence holds up.

Format tips

• Give the numerator and denominator: "24% (7 ÷ 29)" says more than "24%".
• Points aren't percent: going from 30% to 35% is "+5 points", not "up 5%".
• Label partial weeks: "26% · partial · 4/5 engines".
• Leave gaps as gaps: don't fill an unmeasured week with zero.
• Use the same template every period, so stakeholders see change in the data, not in the layout.

On paid plans, in the SOV Tracker dashboard, the competitor set is managed in Competitor Radar under MONITOR, and period reports are in Reports under PROVE.

Rules: invalid answers aren't zero and what we don't claim to measure.

Summary

• An honest summary covers period, scope, engine health, the headline number and its comparability, unmeasured states, work done, what isn't claimed, and the next step.
• Not measured isn't zero: an unmeasured engine or week is never reported as 0%.
• A single query or screenshot is one sample, so answer it with measurement.
• Change isn't cause. Put work done next to the change, but don't claim one caused the other.
• Show numerator and denominator, keep points and percent apart, and label partial weeks.