How we evaluate Wafel with AI

Coding agents read this site alongside people. This page describes what the site does for that reader and how we measure whether it works.

What an agent gets

One corpus, two renderings

Every page is published twice: as HTML for a person and as markdown for an agent. Both renderings are built from the same source, so there is no separate agent-facing copy of the prose. A sentence edited for a human reader is the same sentence an agent acts on.

Markdown is served with the text/markdown content type. A host that serves .md files as HTML causes fetch tools to render the page instead of reading it.

The index is the entry point

/llms.txt is a curated index at the site root. It routes by the shape of a question: looking up an exact name, browsing a group, and summing across everything are different tasks and lead to different pages.

Agents use the index when they have no other way to enumerate the site. In our measurements, runs with filesystem access never opened it and runs over HTTP always read it first. Hosting is the HTTP case.

Agents go to the address they are given and no further. Told that the documentation is at the root, they fetch the root. Given the index, they start there and reach other pages through its links. A line in a project's own instruction file naming the index is therefore the reliable way to route an agent here. Pointing at the index rather than the root does not change answers or cost (measured 2026-09-16 against a frozen corpus). It makes the entry point independent of what the root serves.

Because agents follow the index, its advice has to be safe to follow. The index states which files must be saved to disk rather than read into context, because runs that fetched a large dataset into context failed.

Identifiers have their own addresses

Pages are named for tiers and groups, not for the identifiers they hold. An agent that guesses a page from a tier word lands on a plausible page and finds a name that nearly matches, and the wrong page returns a right-looking answer.

So every token and every construct is also published as a small JSON file, one per identifier, at a URL built from the name. Each file says what the value resolves through and lists the identifiers it can be confused with, since two tiers can carry names that differ only by a prefix and share a value while meaning different things.

Measured 2026-09-14 over HTTP: correct routes on identifier questions rose from 7 of 12 to 12 of 12, answers taken from the wrong page fell from 5 to 0, and token use fell to 0.56×. The index names the route, and every run took it.

Images carry their meaning in text

An agent receives only the alt text of an image. Each image states what it is in its alt text and what it shows in the prose or table cell beside it. The test for an illustration is to cover it and check that the text still makes the point. The same rule for authors is on /how-to-change-this.md.

A DESIGN.md for impeccable

impeccable is a design-quality tool for coding agents. It reads a DESIGN.md at the root of the project it runs in: a frontmatter of colours, radii and type roles that its detector checks code against, and prose that its model-driven commands read.

The site serves one at /DESIGN.md. Its values are read from the same pinned product source as every page here, and its prose is authored in this repository, so the file is regenerated on every build and states its own pin. A project that adopts impeccable vendors a copy and refreshes it when the source is re-pinned. The file links back to the pages here for the full reference, and it is not itself a page: it has no nav entry and no HTML rendering.

Every page states its sources and its gaps

Each page carries its provenance in frontmatter and names the authored file its prose lives in. Where a page has no data for a cell, it says so. An empty cell means we looked and found nothing, which is different from an omission, and only the first is safe to act on.

How we check it

The tools

We test with proctor, which runs an eval, grades it and reports it, and nb, the agent runner it drives. The eval itself lives in this repository beside the content it measures.

The harnesses and models

nb simulates three coding harnesses: Qwen Coder, Claude Code and Codex. Each simulated harness is paired with the model it ships with, hosted by that model's own provider. A harness sets the tool surface, the result formatting and the system prompt, and a model behaves differently in a harness it was not tuned for, so a mismatched pair would measure the mismatch rather than the site.

The run environment

Each run happens in a container that holds no project instructions, no hints and no other network access. The container is given a replica of this site, and the agent reaches it the way it would in real use: it reads /llms.txt, then navigates on its own to the pages it thinks it needs.

The questions

The suite has 14 questions covering the kinds of query that behave differently: point lookup (the value of one named token), fan-out (which things have X but not Y), authored guidance (what a rule says), and discovery (what the site can do at all). Expected answers are computed from the datasets the site is built from and regenerated whenever the corpus is rebuilt, so the answer key cannot drift from the site.

Each question runs three times per harness, because token use and pass or fail vary between runs of the same question. A claim rests on the set of runs, not on one.

The grade records the route as well as the answer: whether the agent used the index, and whether the page it answered from contains the identifier it was asked about. A right answer from the wrong page is recorded as its own outcome, because the route will be wrong as soon as the two pages disagree.

Re-run the evaluation after a significant text change

We re-run the evaluation whenever a significant change lands in the corpus: a rewrite, a reorganisation, a deleted section, or a change to a routing instruction. An agent reads the same prose as a person, so a rewrite can remove the sentence an agent routes on without any human-facing check noticing. A typo fix or a reworded sentence does not need a re-run.

What we have not measured