The verification layer that audited its own liveness (and found its own dead arm)

We run a multi-agent research system that publishes things: research summaries, analysis digests, daily essays on a public site. For about two months we built and measured a claim-verification layer for it — the machinery that is supposed to catch an LLM inventing a number, misciting a source, or quietly inverting a finding. We benchmarked it the boring way: pre-registered decision rules, planted positives, held-out splits, honest nulls reported as nulls.

The headline from all that measurement was unglamorous. Out of a dozen detection ideas — LLM judges, NLI entailment models, decomposition into atomic claims, span-aligned retrieval, multi-arm fusion — exactly one check came out reliably solved: deterministic detection of fabricated numbers by absence. If a number or date in the output doesn’t appear in the grounding source, flag it. No model, no threshold, no vibes. And because of how the routing is built, the guarantee runs one direction by construction: no code path can mark an absent or wrong-valued number as corroborated. That’s not a benchmark score that erodes when the test set changes. It’s a control-flow property you can verify by reading the code.

Everything else came back with honest asterisks. The semantic judge: precision around 0.8, recall 0.2–0.4 on hard benchmarks — and here’s the part that surprised us — swapping in a frontier model barely moved it. Same prompt contract, much bigger model: a few points. Unlocked contract, per-detail auditing: a few more, still below a small local NLI model (F1-macro ~0.62 on FaithBench, which is itself nothing to write home about). The ceiling was the task contract, not model capability. If your roadmap says “verification will get good when the next model ships,” we have measurements that suggest you should check that assumption.

Then, months later, a fresh audit looked at the layer with different questions. Not “how accurate are the arms?” — we knew that. The questions were: are the arms actually alive, and are they pointed at the right surfaces?

Finding one: enabled is not alive

Our NLI arm runs in an isolated virtual environment (torch stays out of the production environment; the host shells out to a scorer and reads one JSON line back). Eleven days before the audit, a destructive git clean had wiped that environment. It was gitignored, so version control couldn’t restore it, and nothing had exercised it since — the surface it serves was dormant.

The config said the arm was enabled. The code was correct. Every future invocation would have failed gracefully — a typed nli_verdict: ERROR, "venv or scorer missing" — exactly as designed. And that’s the trap: a well-engineered fail-open arm is silent in the way that matters. No crash, no alert, no rows. An enabled arm that cannot run and a healthy arm with nothing to do produce the same evidence: nothing.

The audit caught it by ignoring the flags entirely and reading the stores. Which log files are receiving rows, and how fresh are they? Two arms: rows from today — alive. One store: absent from the entire repo — either the surface is dormant or the arm is dead, and you’d better find out which. (Both, it turned out.)

Two rules fell out of this. First: audit stores, not flags — configuration tells you intent, only output tells you liveness. Second: fail-open must be typed and counted. Our ERROR dict meant the failure was at least classifiable the moment someone looked; if the arm had returned a default verdict instead, the dead environment would have been indistinguishable from working verification forever. An abstention you can count is an SLA metric. An abstention that looks like a verdict is a lie with good posture.

Finding two: the map was inverted

The more interesting finding wasn’t a dead component. It was that the layer’s deployment map had the reliable check pointed at the wrong surfaces.

Recall the one solved check: numbers absent from the grounding are flagged. Its weakness is equally simple: its precision is not a property of the checker — it’s a property of the surface’s grounding. Run it on an open-world document (a research paper that legitimately cites the broader literature) and it flags every real-but-uncited statistic. We measured ~3% precision on that class of surface. Same checker, closed-world surface — output generated from nothing but a known prompt or source set — and the grounding is complete by construction, so a flagged number is almost certainly invented. The checker didn’t change. The surface did.

Our map: three verification arms armed on the open-world paper surface (where the reliable check over-fires and everything runs advisory) — a surface that had been dormant for weeks. Meanwhile the system’s most public surface — a daily LLM-written essay, auto-published to the live site, 71% of existing pieces carrying number or date tokens — had no verification at all. And a digest pipeline sat in between, its enforcement gate blocked for two months on a measured “fabrication rate” of 0.79 that was actually a coverage artifact of the extraction step, not hallucination.

The fixes were almost embarrassingly cheap once the frame was right:

• The public essay surface got the closed-world check the same day: every number/date in the output is checked against the generation prompt. Deterministic, zero LLM calls, wired with an advisory posture (log + a frontmatter stamp + a push notification to the operator) because we don’t enforce anything whose on-domain fire rate we haven’t measured.
• The digest gate got a pre-registered re-measure on the closed-world basis: check the digest’s numbers against the raw source items instead of extracted claims. Result: 0.026 fire rate vs the old 0.79, roughly 2% tokenizer over-fire across every flag we hand-inspected — and the single flag on the best-grounded digest was a genuine catch, a cited arXiv identifier that appeared in no source item. The gate that looked unfixable was mostly measuring its own extraction coverage.

The rule that generalizes: classify a surface closed-world vs open-world before wiring verification onto it. Closed-world surfaces (summaries, digests, prompt-grounded generation) can take the cheap deterministic check immediately, and it’s precise there by construction. Open-world surfaces need the grounding-completeness conversation first, and anything you wire should start advisory with a measured over-fire rate before it blocks a single publish.

What we’d tell anyone running agents in production

1. A check’s precision lives in the surface, not the checker. Before asking “how good is our verifier,” ask “does this surface’s grounding cover its claim space?” The same detector is a 97%-precision fabrication catcher on one surface and a noise cannon on another.
2. Enabled ≠ alive. Every verification arm, monitor, or guard you own should be auditable by its output store’s freshness, not its config flag. Schedule that audit; ours found an eleven-day-dead component in an afternoon.
3. Fail-open needs a type system. Degenerate inputs (empty evidence, zero extracted claims, missing dependency) must produce a counted, classified abstention — never a verdict. We learned this twice the hard way before it stuck: an empty claim set once made the verifier flag everything as fabricated, which polluted the very metric used to decide whether the gate was safe to enforce.
4. By-construction guarantees outrank measured rates. “0/883 planted wrong-value numbers laundered” is a nice benchmark row. “No code path can mark an absent value corroborated” is a property an auditor can verify by reading the routing, at any n, on any distribution. Where you can design the safety in, do that instead of certifying it.
5. The model-upgrade lever is usually not the lever. Measure before you wait for a bigger model. Ours would have changed nothing about any finding in this post.

What this does not solve

Honesty about the boundary, since the whole point is receipts. Semantic and entailment-level faithfulness remains genuinely hard: our best local arm sits around F1 0.62 on a hard benchmark, judge recall on subtle meaning errors is 0.2–0.4, and nothing we tested — including frontier models and every decomposition scheme in the recent literature — cleared that class. Derived numbers (“revenue doubled,” “a 12% rise”) are an open alignment problem; our pipeline abstains on them, visibly, rather than pretending. And a present number with inverted meaning (“rose” where the source says “fell”) needs its own arm — ours exists, is advisory, and its production efficacy is unmeasured, which is exactly what its documentation says. The reliable core is narrower than a product brochure would like: invented numbers, absent citations, wrong values against a grounded source — caught deterministically, with a guarantee. The rest is bounded, labeled, and priced as what it is.

That narrowness is the finding. Most of what a verification layer earns you isn’t a magic detector — it’s an inventory of what’s actually solved, what’s advisory, what’s dead, and what’s pointed at the wrong thing. Our layer’s most valuable output this month wasn’t a verdict on a document. It was a verdict on itself.