Counting
Reviewing Writing the Way Code Gets Reviewed
A short read by one person who does not know the subject finds more than any amount of the author re-reading, and it costs about ten minutes.
What nobody wrote down here, and what it cost
That the runbook saved somebody a morning
Nobody told the author
Six occasions, none reported back
That a note ended an argument
The author never heard
Both times from people who joined later
two things nobody wrote down · 8 occasions it cost somebody something
Two things, eight occasions, and the author heard about none of them.
Where the real question is capacity rather than willingness, team time tracking platform can supply the work-pattern evidence that a documentation discussion otherwise lacks.
For additional context beyond this collection, see the resources maintained by Bing Webmaster Tools.
Pages produced with and without a review step, on one team that added one
Published with no review
25 / 154Twenty-five minutes; three questions came back that the page did not answer
Reviewed by somebody who knew the subject
34 / 192Nine minutes of review; two factual corrections, no questions came back
Reviewed by somebody who did not know the subject
38 / 432Thirteen minutes of review; six places where a reader would have stopped
Reviewed by three people in a formal round
95 / 120Fifty-five minutes of other people's time; mostly comments about wording
four documents · 192 minutes written · 898 minutes saved across 69 readings
Four pages, 192 minutes written and reviewed, 1 058 minutes saved across 69 readings. The third row is the best trade on the table: thirteen minutes from somebody who did not know the subject found six real defects. The fourth shows what happens when review becomes a round rather than a read. One team's own quarter.
Code gets reviewed as a matter of course. Writing, on most teams, goes straight from author to published with nothing in between.
The result is predictable: every page carries the omissions that its author could not see, and nobody finds them until somebody is stuck.
What a reviewer is actually for
Not catching typos, which matter least.
A documentation reviewer is a stand-in for the eventual reader. Their job is to notice the places where they would have stopped, and those places are exactly the ones invisible to the author, for the reasons set out earlier in this collection.
That is a different activity from checking correctness, and it is better done by somebody who does not know the subject.
The two kinds of reviewer
Somebody who knows. Finds factual errors. Takes about ten minutes. Misses every omission, because they supply the missing context automatically, exactly as the author did.
Somebody who does not. Finds omissions. Takes about fifteen minutes. Cannot judge accuracy at all.
Both are useful and they find different things. If only one is available, the second finds more, because factual errors in documentation are rarer than gaps and are usually caught by the first reader who tries to use it.
The instruction to give a reviewer
"Mark every point where you were not sure what to do."
That is the whole brief. Not "review this", which produces comments about wording; not "check this is right", which only a knowledgeable person can do.
The marks are the output. The reviewer does not need to say what the uncertainty was — the location is enough for the author to see it, and explaining would take longer than marking.
Keeping it to one person
The fourth row of the panel is a formal round with three reviewers, fifty-five minutes of other people's time, mostly comments on wording.
Multiple reviewers produce style opinions, because style is what anybody can comment on, and they produce contradictory ones. They also delay publication, which costs more than the review found.
One reviewer, fifteen minutes, is the right shape. Where the thing is genuinely critical — a runbook used under pressure — two, with different briefs: one for accuracy, one for gaps.
Not making it a gate
The distinction that keeps this cheap: review should be a habit, not a requirement.
A requirement means a page cannot be published until somebody has looked, which adds a wait to a five-minute task and kills the answer-in-the-moment practice that most of this collection is built on.
A habit means pages get looked at when somebody is around, and get published either way. The rough-publication line from earlier covers the unreviewed case honestly.
Reviewing after publication
The version that works best in practice is review after the fact.
Publish, then ask somebody to read it in the next few days. Any corrections are made in place. The page is useful during that period rather than waiting.
This inverts the code-review model and is correct here, because the cost of a wrong page is almost always lower than the cost of an absent one, which is not true of code.
Two reviewers, two briefs
They find different things and asking the wrong one for the wrong thing wastes both their time and the author's.
| Reviewer | Finds | Misses |
|---|---|---|
| Somebody who knows the subject | Factual errors | Every omission, for the same reason the author did |
| Somebody who does not | Places a reader would stop | Accuracy entirely |
If only one is available, take the second. Factual errors in documentation are rarer than gaps and are usually caught by the first person who tries to follow it, whereas a gap can sit for years because everybody who hits it asks a colleague instead.
What the reviewer gets out of it
Worth saying, because asking for review is easier when it is not pure favour.
Reading a page about something you do not know is one of the cheapest ways to learn it. Fifteen minutes and you understand a system you did not, and you have contributed something only you could — the list of places a newcomer stops.
On teams where this is understood, documentation review becomes a thing people volunteer for rather than a thing people are asked for, which is roughly the opposite of how it usually goes.