Audience metrics and what they cannot tell you

1. Defining the units before reading the chart

Newsroom dashboards mix units that are not comparable. A pageview counts a request. A session groups requests within an inactivity window that the tool defines, often thirty minutes, so one reader can generate several sessions in an afternoon. A unique user is an estimate derived from stored identifiers and is therefore inflated by device switching and deflated by shared devices and cleared storage. Attention time is measured only while the page is in view and the tab is active, and different tools apply different idle thresholds. Two dashboards can disagree by a wide margin while both being correctly implemented. Comparisons across tools require the definitions, not just the numbers.

2. Attention time is better, not sufficient

Attention time is a genuine improvement on pageviews because it distinguishes an item that was opened from one that was read. It is still not a measure of value. Long attention can indicate absorption or a badly structured piece that forced re-reading. Short attention can indicate abandonment or an efficient answer to a question the reader actually had — a transport alert that takes eleven seconds has done its job. The remedy is to set expectations by format before publication: state what a successful outcome looks like for a briefing, a feature and an explainer, and judge each against its own standard.

3. Inflation, bots and referral noise

A meaningful share of raw traffic is not human. Crawlers, monitoring services, prefetching, link previews and automated scrapers all generate requests, and referral spam appears in reports without ever reaching the site. Sudden traffic spikes with near-zero attention time, unusual geographic distributions or implausible referrers should be excluded before anyone reports a record week. The practical safeguard is a documented filtering rule applied consistently and reviewed periodically, plus a habit of checking whether an exceptional figure is matched by a corresponding rise in a harder-to-fake indicator such as newsletter opens or returning readers.

4. Cohorts, loyalty and the useful question

Aggregate totals hide the distinction that matters most: whether the audience is the same people returning or different people arriving once. Cohort analysis groups readers by when they first arrived and tracks what proportion return in each subsequent week. That single view answers questions totals cannot — whether a widely shared piece brought anyone who stayed, whether a new format built a habit, whether a section is retaining or churning. For most publications, the number of readers who return at least weekly is a better health indicator than any monthly total, and it moves slowly enough to be trusted.

5. Incentives created inside the newsroom

Metrics shown on a screen in the office become targets whether or not anyone declares them. Ranking reporters by pageviews reliably produces more of whatever generates pageviews, which is rarely the coverage the publication says it exists to provide. Desks that manage this well separate diagnostic use from evaluative use: metrics inform decisions about headlines, timing, format and promotion, but individual performance is assessed on accuracy, sourcing, difficulty and public value. Where a leaderboard exists at all, it should be at the level of the desk and should include at least one measure that volume cannot satisfy.

6. Rituals that make measurement useful

Data changes decisions only when there is a moment for it to enter them. A short weekly review that looks at five things — best-performing item and why, worst performer that should have worked, one format experiment, returning-reader trend, one correction or complaint — is more effective than continuous dashboard monitoring, which mainly produces reactive behaviour. Write the conclusions down. The value of the ritual is the accumulated record of what the desk expected, what happened and what it changed, which after a few months becomes genuine institutional knowledge rather than an anecdote about last Tuesday.

7. Measurement without surveillance

Useful measurement does not require tracking individuals across the web. Server logs give requests, referrers and rough geography without any client-side script. Aggregate, cookie-free analytics can report page-level totals and referral sources with no persistent identifier. Newsletter platforms report opens and clicks against an address the reader gave deliberately, though open rates have become unreliable as mail clients pre-fetch images. Under the GDPR and ePrivacy rules, minimising what is collected reduces both compliance burden and the risk of holding data a publication cannot justify. This site itself installs no measurement tag at all.

8. Reporting figures honestly

Audience figures published externally should state the metric, the tool, the period, the filtering rule and any change in methodology since the last statement. Switching definitions between reports — from sessions to users, from a month to a rolling average — makes a series meaningless even when every individual figure is accurate. If a comparison is affected by a single exceptional event, say so. The same standard the newsroom applies to figures supplied by an organisation it is reporting on should apply to the figures it publishes about itself.

Metrics and their proper use

Every metric in this table is defensible in its own column and misleading outside it. The failure is not choosing the wrong metric but using a good one for the wrong claim.

Metric hygiene

• Each metric in regular use has a written definition and a named tool.
• Filtering rules for bots and known crawlers are documented and reviewed.
• Format-specific expectations are set before publication, not after.
• Returning-reader trend is reported alongside every total.
• Individual performance is not assessed on volume metrics.
• A weekly review with written conclusions replaces continuous dashboard watching.
• Externally published figures state metric, tool, period and filtering rule.
• Methodology changes are flagged in the first report that uses them.
• Collection is minimised to what the publication can justify holding.

Re-run the list whenever a measurement tool is changed. Most broken audience series begin with a migration nobody documented.

Questions readers ask