Skip to main content

The Drift Race: methodology and measured results

We published three predictions before running the experiment. Here is the method, the scoring, and what the run returned — including the one we got backwards.

Broken references

Raw
19
RIFT
0

Nine dead paths, two dead fragments, eight unreachable stylesheets. RIFT finished with none of the three.

Style forks · we predicted this wrong

Raw
9
RIFT
28

We expected RIFT to win here. It lost, in both governed runs — and 22 of the 28 arrived in a single operation.

Tokens used

Raw
14.1M
RIFT
8.5M

39% less for the same thirty operations. Structure removed work rather than adding ceremony.

01020 3040 ADDREORGANISERETIRE ENRICHCONSOLIDATE peak 44 19 0 ■ raw files ■ RIFT
Broken references present in the published site after each operation — every point an independent audit of the whole site, not a running total. Both arms started from the same site, which already carried one.

Adding is safe. Moving is what breaks a site.

For eleven operations the two sites are indistinguishable. Then comes the first reorganisation that moves pages other pages point at, and the raw site never recovers — not even after a final operation whose only job was to find and fix broken links.

In raw files a link is a literal path, so moving a page silently invalidates every reference to it. In RIFT a link is a reference to a page, so moving it repoints every inbound link automatically.

The residue is specific. Eight of raw's dead links point at two pages — a glossary and a support page — from four concept pages each. Those two moved, and nothing that pointed at them was told.


The failure a review would not have caught

The worst damage in the raw run was not a missing page. It was this, written into eight pages:

<link rel=\"stylesheet\" href=\"/assets/ds/3d67e511-….css\"

Backslashes where quotes belong. The stylesheet never loads. One page in six was serving with no design system at all — and still returning 200. Those eight pages are the difference between raw's eleven broken links and its score of nineteen. RIFT rejects that markup at the moment it is written, so the governed run could not produce it.


Where we were wrong

We predicted RIFT would reduce style forks. It tripled them — 28 against raw's 9, in both governed runs. The published snapshots say precisely where they came from.

Through twenty-eight operations the governed site carried six. Operation twenty-nine took it to twenty-eight. That operation asked for one thing:

Add a "Last reviewed: 2026-09-01" line to every page
in the API Reference section, formatted consistently.

The design system had no component for a review stamp. So the model wrote one by hand, eleven times — once per API page — and each copy carried its own inline style rule and its own hard-coded colour. Two forks per page, eleven pages, twenty-two forks in a single operation.

Governed forks by rule, before and after that one operation

inline style2 → 13
hard-coded colour1 → 12
unknown class1 → 1
raw table2 → 2

Governance stops a model writing a broken link. It does not stop a model inventing styling it cannot find. That is a gap in the design system, not in the governance — and the fork count is better read as a measure of design-system coverage than of governance.

Removing the twenty-two forks attributable to that one operation would leave the governed run at six against raw's nine. We are not claiming that as a result: it did not happen, and the missing component is our responsibility. But it does locate the fix — one component, not a change to how governance works.


What the test measures

After every one of the thirty operations, the whole published site is crawled and audited from scratch. Not a diff against the previous version — an independent examination of the site as it stands, so a fault introduced at operation twelve and never repaired is still counted at operation thirty.

Seven measures are recorded. Three are reported here because they are the ones a content team feels: broken references, style forks and token cost. The others cover header and footer consistency, pages nothing links to, how many pages each change disturbed, and whether the change that was asked for happened at all.

That last one matters more than it sounds. An arm that quietly skips an operation damages nothing, and would otherwise score beautifully. So each operation carries a check written from its own instruction — a page that was supposed to move is confirmed moved — and an arm cannot look good by doing nothing.

See it for yourself

  • github.com/heitham/godzilladocs — the site under test. Each run is its own branch, and every operation left a commit, so you can check out any point in a run and read the site exactly as it stood, broken links and all.
  • github.com/heitham/Drift-race-godzilla-method — the harness, the frozen operation list, the scoring code and the per-operation results.
  • The scorer is substrate-blind and re-runnable: tsx scoring/score-run.ts <run> regenerates every number on this page from the snapshots without re-running a model.
  • Found a flaw in the method? That is the point of publishing it. We found five ourselves while running it.

Why we published the losses

A vendor benchmark that reports only wins is marketing. This one predicted three outcomes in public, measured them, and got one backwards. The style-fork result is on this page because it is true, and because a measurement built so it cannot lose measures nothing.


Method

  • One 30-page documentation site, 129 internal links, published from RIFT.
  • Thirty operations, written and frozen before any trial: add, reorganise, retire, enrich, consolidate.
  • Each operation runs in a fresh session with no memory of the last — a site changing hands, not one continuous mind.
  • Both arms get the same model, the same instructions and the same design-system reference. Only the substrate differs.
  • Every operation is checked for whether the requested change happened, so neither arm can score well by doing nothing.

What this does not prove

  • One model, one site. Directional; no statistical significance is claimed.
  • RIFT gained a write-time link validator during the experiment, because the experiment exposed the gap. Part of that zero is the validator refusing bad input.
  • RIFT could not retire a page during these runs, so it left behind content it was asked to remove. Counted against it above.
  • Raw completed 27 of 30 operations, RIFT 29 — an arm that skips work damages less, which is why completion is scored.
  • A second model is running now. The method, operations, scoring code and raw results are published in full.

Run configuration

Claude Haiku 4.5, identical reasoning configuration in both arms. Reported pair: raw-v2 against governed-v8. The governed result reproduced across two runs on two different publishing paths. Every number is recomputable from the published snapshots without re-running a model.