How it works

Three stages, deliberately kept apart: capture what happened, store it so it cannot be edited, and interpret it separately so the interpretation can be thrown away and redone.

1. Capture happens on the server

Nearly all web analytics runs as JavaScript in the visitor's browser. That design is blind to the population we care about, because crawlers generally do not execute JavaScript. A tracker that only reports when scripts run will faithfully tell you that no crawlers ever visit.

So capture here happens in the request path itself. When any client asks for any URL, the server records the method, host, path, query, HTTP version, the complete set of request headers, the client address, the response status, the bytes served, and the time taken. No consent to run code is required, because no code needs to run.

Client-side JavaScript still exists on exactly one page, the JS probe, where its execution is the thing being measured rather than the means of measuring.

2. Observations are immutable

Records are written once and never updated. They contain no score, no verdict, and no label — no field that says "bot" or "suspicious". Those are conclusions, and conclusions belong to the next stage.

The separation matters because a stored conclusion is unfalsifiable later. If the record says a visitor was a bot, you can never re-examine that call; the evidence has been overwritten by the judgement. Keeping the observation clean means every conclusion drawn from it stays reviewable.

3. Interpretation is versioned

A request is recorded once and never edited. Every published figure is derived from that record by an interpreter carrying a version number, so improving the method means recomputing history rather than starting a new dataset.

Signals — automated enumeration, robots violations, JavaScript capability, and so on — are derived from the stored observations by an interpreter that carries a version number. Its output is written to a separate table tagged with that version.

When the interpreter improves, the previous version's output is deleted and recomputed against the same untouched observations. Nothing is estimated and nothing is backfilled from memory. If a conclusion cannot be re-derived from the record, it is not a conclusion we will publish.

What this buys

Every number on the lab page can be traced back to specific observed requests. When a method turns out to be wrong, only the interpretation is wrong; the evidence survives, and the corrected reading can be computed over the full history rather than only over data collected after the fix.