---
name: telemetry-that-fires-only-on-failure-cannot-witness-a-success
description: "A record emitted only on the error path leaves the successful path unwitnessed — so the ONE question a post-mortem asks (what ran, in which generation) is exactly the one the log cannot answer; and a mutex that serializes within a process proves nothing about two processes"
metadata:
  node_type: memory
  type: feedback
---

Measured 2026-09-09 triaging a double-serve at B (#272 golden r2 a4, REST stream 85 served twice
25.3 ms apart). Two findings worth keeping, both about the SHAPE of evidence rather than that bug.

**1. Failure-only telemetry.** `dispatch.rs` emits `DISPATCH_EV` on non-Served outcomes
(`:801-816`), breaker-trip (`:826-838`) and oneway-retire (`:858-871`) — a CLEAN serve emits
nothing. B's whole log therefore carried `grep -c DISPATCH_EV` = **0**, so the dispatcher's
generation id, attempt count and conn were unrecorded on precisely the path that ran. The
instrument existed, was well designed, and was silent exactly where the question was.

**2. A mutex proves a PROCESS, not a system.** The claim map (`:746` insert under lock, `:455`
`should_claim`, `:876` release after the worker returns, 500 ms retry floor at `:346`) makes a
second serve impossible *within one dispatcher* — and I nearly wrote "therefore impossible". It is
one map per dispatcher, one dispatcher per brain child process (`brainproc.rs:432-441`,
`consumer_gate :418-420` = spawn-once-PER-PROCESS). Two processes, two maps, no shared exclusion.

**Why:** both errors flatter the reader. Failure-only telemetry makes a quiet log read as "nothing
unusual happened"; a per-process lock makes a source read produce a confident "cannot happen" that
the log contradicts. Where they meet, the honest answer is a mechanism you can only INFER, and the
temptation is to pick the reading that needs no gap.

**How to apply:** when a post-mortem needs to know WHICH instance did something, check first
whether the instrument fires on the success path at all — `grep -c` the token before reasoning from
its absence ([[empty-inbound-is-never-material]] is the same trap on a different surface). When a
lock or map "makes it impossible", name the scope the lock actually has (thread / process / node)
and ask what a second instance of that scope would do. And when the evidence runs out, say so as
its own numbered point instead of letting the conclusion cover it — see
[[a-source-read-cannot-witness-an-integration-cell]].