diff --git a/docs/INFRA-REGISTER.md b/docs/INFRA-REGISTER.md index 5daa7951..6285a0d2 100644 --- a/docs/INFRA-REGISTER.md +++ b/docs/INFRA-REGISTER.md @@ -4860,6 +4860,17 @@ the wrapper's harness children orphaned + running") — so IR-80 ships callers, a comparable pair. **Arm 1 stays OPEN**, its re-run deferred until **IR-90**'s guard exists and a window with no `main` push and no CI job on the box can be scheduled. +- **AMENDMENT 2026-09-10 (hertz, on THIS ENTRY'S OWN arm-1 design note).** The note says the arms run + **INTERLEAVED** A/B/A/B so box drift cancels. It does not cancel: **B is always the later arm within + each pair**, so a monotone drift across the session lands on B every time and is read as an arm + effect. Alternate the PAIR ORDER — **A/B then B/A** — record per-leg timestamps and conditions, and + describe the result as **REDUCING order confounding, not eliminating it**. Second binding change, + from the releases#294 triage: **instrument, do not infer.** `cargo nextest run --success-output + immediate` retains stdout for PASSING tests (deployah), so a passing arm can still print poll + elapsed, iteration count, IPC time and producer progress — **a failing arm is not required** for a + margin measurement (todlando, correcting deployah's "it has to fail to be informative"). Arm 1 was + never re-run, so this lands before the measurement, not after it. + ### IR-86 — golden's 32 GiB floor is BELOW the measured 67.4 GiB Windows suite footprint, so a start floor passes a box that cannot finish - **Status:** OPEN, filed by doyle 2026-09-08 at the #272/v0.68.0 golden r2 triage. · **Origin:** @@ -4928,6 +4939,30 @@ the wrapper's harness children orphaned + running") — so IR-80 ships callers, - **Ripe when:** next `golden.yml`/twohost rig touch. · **Size:** small-medium (job shape or barrier rework + panic text). +- **AMENDMENT 2026-09-10 (releases#294, golden 34445961595 att1 @ `a9e786b2`) — THE STARVING RUN CAN + BE GOLDEN'S OWN PR, which makes this self-inflicted and recurrent on every golden run with a PR + open.** Measured timeline: **07:18:59Z** golden `test (Windows)` completes and releases hfenduleam; + **07:18:59Z** twohost-a queued; **07:19:01Z** PR run **34446095808** job `unit (Windows, + hfenduleam)` SEIZES the runner; **07:19:02Z** twohost-b starts on the free kitsubito; **07:36:16Z** + twohost-b fails at **900.41s** — `never converged on the rig: pairing: A rostered via the + daemon-hosted responder`. The origin case (2026-09-08) was a post-merge push on `main`: a run + belonging to someone else. **Here the starving run is the `pull_request` run of the very PR carrying + the golden head**, so opening the golden PR arms the starvation of that same head's twohost pair. + **Consequence for the remedy shape above: any option that only de-conflicts against OTHER lanes' + runs does not reach this case.** The golden head's own PR has to be excluded from the Windows + runner for the duration of the golden run (or the pair collapsed into one job, option 1). +- **SECOND-ORDER FACE (deployah, same run): starvation does not merely red B — it desynchronises the + pair past repair.** twohost-b was TERMINAL at 07:36:16Z; twohost-a did not start until **07:48:07Z** + — the two halves never overlapped for a single second, so twohost-a's outcome is uninformative + **whatever it says**, and a red there must not be counted as a second product failure. Recorded + honestly at deployah's own insistence: **VOID is his SCORING interpretation, not a fact about the + run** — the release record carries GitHub's raw `conclusion` for both jobs beside the scoring, not + laundered into it. +- **⚠ ITS WALL CLOCK IS INDISTINGUISHABLE FROM [[IR-89]]'s.** A 900 s twohost red whose panel blames + pairing is ALSO the exact signature of the banked Windows-firewall hazard. The panel alone cannot + separate them; **the job TIMELINE can**. Check the schedule before accepting the firewall hazard for + any 900 s twohost red — this one fit that signature perfectly and was not it. + ### IR-88 — `pool-release` via `cargo run -p xtask` REBUILDS xtask INTO the pool it is releasing, so a reaped pool regrows ~2.8 GB in silence - **Status:** OPEN, filed by doyle 2026-09-08 from hertz's reap report (20:00Z) during the #272 @@ -5262,6 +5297,49 @@ the wrapper's harness children orphaned + running") — so IR-80 ships callers, `spt-daemon` (todlando), test-side legs (hertz). · **Origin:** a 22:35Z empty `` turn trigger after the #292 daemon restart, chased for its body; the body was in a spill file, and the chase exposed that neither agent's io-events log held the messages that woke it. +- **THE TWO PUBLISH SITES ORDER THEMSELVES OPPOSITELY — read at `de5a44bc`, 2026-09-10, from the + bare-baseline red on `io_events_undriven_kinds_e2e` (golden 34467873057, Linux Phase A).** The + count below is right and the ORDERING is what this entry was missing: + · **PATH A, hook poll** (`delivery.rs`): `poll_drain()` publishes EVERY delivered row (`:757`) and + RETURNS them; its only caller `cmd_poll` (`:660`) then loops `emit_event` over the returned rows. + **Publication happens-before presentation**, strictly. An immediate log read after observed + presentation is CONTRACT-CORRECT here, and a bounded retry on this path WOULD conceal a real + violation. **Leave Path A alone.** + · **PATH B, the relay** (`startup.rs:1057`, the `deliver` closure): `emit(id, from, body)` — the + presentation — runs, and `publish_msg_io(...MSG_IN...)` runs AFTER it, per message, in the same + closure; the closure's own comment records that "Both backlog (after notif validity) and live TCP + reach this edge." **Presentation happens-before publication.** + A test that detects presentation by reading the listener child's STDOUT through a pipe and then + reads the io-events log immediately is therefore **racy BY CONTRACT on Path B** — publication for + that message runs after its emit, in the child, and the pipe read can win. A bounded wait on the + log is permitted on Path B: **it does not weaken any publication-before-presentation guarantee + there, because there is none** (hertz's and todlando's narrowing — NOT "conceals nothing": the + wait still DEFINES WHICH PUBLICATION DELAYS THE TEST ACCEPTS, and it does not establish why any + given execution missed a row). A timeout would establish "not published within the bound", never + "never publishes", and cannot alone identify this entry's defect. +- **A SECOND, INDEPENDENT DEFECT IN THAT CELL'S ASSERTION, true regardless of the above.** + `io_events_undriven_kinds_e2e.rs:818` asserts exact-once as + `vec![("MSG_IN", Some(AUTHOR)), ("MSG_IN", Some(AUTHOR))]` — a `(kind, author)` pair per row, and + **both rows are identical**, so the assertion cannot tell the backlog row from the live row and + **two BACKLOG rows would satisfy it**. The identities exist and are already stamped: + `publish_msg_io_local` carries `.with_msg_id(msg_id_of(body))` — the short ID the failing panel + itself printed. Exact-once belongs per MESSAGE ID, not per author. + **AND IT CUTS BACKWARDS, WEAKENING EVERY GREEN THIS CELL HAS EVER RETURNED (deployah, on + arm-comparability grounds — the point nobody else had drawn):** the three PASSING executions of + this cell in the instrumented pair (base linux 3.612 s, head linux 2.343 s, head windows 16.411 s) + did **NOT** establish that both relay arms published. They established that two rows matching + `(MSG_IN, alice)` were present at read time. A green here is a WEAKER fact than it reads as — the + verdict stands in for a measurement the assertion never made. So "both Linux legs were green + across the instrumented pair" does not become FALSE, it becomes NARROWER than it was being used + for: it is a statement about the ASSERTION's outcome, not about publication. It does **not** make + the bare red likelier to be a product defect, does not classify anything, and does not separate + the race from non-publication — it removes a comparison the RCA was implicitly leaning on. + (Raised independently by hertz and todlando as an assertion defect; its backwards consequence for + the greens is deployah's.) +- **Status of that red: UNEXPLAINED between DELAYED and ABSENT publication.** Path B makes the race + MECHANICALLY AVAILABLE, which is more than "not excluded" — but availability is not occurrence, + and one observation does not separate them. Kinship to this entry stays a HYPOTHESIS. The cell is + in neither `docs/FLAKE-LEDGER.md` nor this register (0 hits each): an unregistered signature. - **The mechanism, read at `de5a44bc`:** `MSG_IN` has exactly two publish sites in the tree — `crates/spt/src/api/delivery.rs:757` (`publish_msg_io_local`, reached only from `poll_drain`, the hook-poll leg, `TakerLeg::HookPoll`) and `crates/spt/src/api/startup.rs:1064` (`publish_msg_io` @@ -5336,3 +5414,368 @@ the wrapper's harness children orphaned + running") — so IR-80 ships callers, - **Ripe when:** any lane touches `spt-daemon::inject` or the IO funnel; small enough to ride the next product batch. · **Size:** one publish call at one chokepoint, one unit cell, one `int` leg (a hook-hosted receiver woken by a `SENT`, then `io-events` replayed for the msg-id). + +### IR-93 — no PDB ships beside the installed `spt` binary, so the one non-invasive read of a wedged production daemon comes back without a single named frame + +- **Status:** OPEN, filed by doyle 2026-09-10. **The number was reserved by NAME on 2026-09-09** at + the #293 mechanism read ("PDB ask -> IR-93 at next sweep") and this entry closes that claim — a + name-only IR with no entry text is the exact gap [[IR-85]]'s status line had to spend a paragraph + explaining about IR-84, and it is not being left open a second time. · **Origin:** the #293 pump + wedge on hfenduleam. +- **What was actually paid.** With the daemon wedged and unrecoverable, hertz's `cdb -pv -p 31856` + dump (2026-09-09 23:23:26-31Z, 4.79 s suspend, 35 threads, preserved at + `.spt/preserved/293-brain-stacks-20260909T232326Z/`) was the ONLY direct observation available of + the live wedged process — the pump had already stopped, so the suspend cost the fleet nothing it + still had. It came back **with NO SYMBOLS: there is no PDB beside the installed exe.** Both + pump-ipc-reader threads were visible in `SleepEx` and all 32 brain threads in Wait, but the frames + are addresses. +- **The cost, stated at the strength it holds.** The mechanism was named anyway — from SOURCE reading + (`Brain::read_event()` unbounded at `brain.rs:1025-1027`, 17 in-brain loopers), not from the dump. + What the dump could not do is say **WHICH call site parked THAT instance**, and that remains NOT + ESTABLISHED. A symbolised dump is the one artifact that answers it directly, and the next wedge + will present the same choice with the same empty answer. +- **Remedy:** retain and ship the PDB beside the installed binary (or publish a symbol store the + debugger can resolve against) so a production dump names frames. **Weigh it honestly at rider + time:** this is a packaging/size decision, and a release profile that emits no debug info is a + deliberate choice someone made — this entry is the bill for it, not a claim it was wrong. +- **Kin:** [[IR-82]] (a daemon death leaves no record of who died — same family: the corpse is the + evidence and nothing preserves it), [[IR-84]] (the arm with no clock), [[IR-95]] (a budget that + cannot say how much it burned). +- **Ripe when:** next installer / release-packaging touch. · **Size:** packaging (profile + install + layout), no product code. + +### IR-94 — xtask's brainread deadline gate cannot SEE `serve_sync`: its registry is three hand-listed files and its predicate is a name shape, so a stream server outside both got the reply form and the gate stayed green + +- **Status:** OPEN, filed by doyle 2026-09-10 at the releases#294 / v0.69.0 golden RED triage. **This + is an INDEPENDENT gate finding, not the cause of that red** — it was found while reading the + milestone's diff and stands whatever Finding 1 turns out to be. · **Origin:** + `xtask/src/brainread.rs`, new this milestone (the #293 fix's zero-call-site guard). +- **Mechanism, read at source.** The gate's REACH is two hand-authored narrowings stacked: + **registry = exactly three files** (`dispatch.rs`, `attach.rs`, `brain.rs`) and **predicate = a name + shape** (`starts_with("serve_") && ends_with("_feed")` or `== "serve_attach"`). `serve_sync` is a + stream server that lives in **no registry file** and whose name **escapes the pattern**, so it is + invisible on both axes at once. It therefore carries the **REPLY form — the exact form this gate + REJECTS for every stream server it can see — and the gate is GREEN.** An enforcement whose + population is enumerated by hand reports compliance over the set it can see, which is not the set + that matters. +- **Why it is inert TODAY and why that is not reassuring.** The production carrier is unbounded: + `dispatch::connect()` (`dispatch.rs:898`) uses `Brain::cold_start` with `io_timeout: None`, so + `call_deadline()` is `None` and the read is unbounded regardless of which form the server carries. + **It becomes LATENT the day `serve_sync` rides a pump carrier** — which is precisely the direction + the #293 fix is moving the codebase. The guard will still be green on that day. +- **Owned:** todlando has taken that his migration/census missed `serve_sync`'s lifetime semantics. + Recorded because the entry is about the GATE's reach, not about who missed it. +- **Remedy shape:** derive the population from the **behaviour** (a fn that serves a stream) rather + than from the file it sits in or the name it happens to carry, and make the gate **FAIL on a server + it cannot classify instead of skipping it**. Add a census leg that PRINTS the population checked — + a count is what would have exposed a three-file registry at a glance. +- **Kin:** [[IR-84]], `unbounded-brain-carrier-cannot-be-bounded` (memory — the carrier question that + makes this inert today), `REQ-BRAIN-READ-BOUNDED-PER-CALL`. +- **Ripe when:** next `brainread`/xtask touch, **and mandatorily BEFORE `serve_sync` is migrated to a + bounded carrier.** · **Size:** small. + +### IR-95 — a burned polling budget names WHAT it wanted and never HOW LONG it waited, so no run, green or red, can say how close it came + +- **Status:** OPEN, filed by doyle 2026-09-10 at the releases#294 golden RED, where it cost this + triage most of a morning and four agents. · **Origin:** the two Windows Phase A failures at + `a9e786b2`, both polling budgets burned rather than value mismatches. +- **The two sites, read at source.** They are not even the same KIND of budget, which is the first + thing no reader can tell from the panel: + · `crates/spt/tests/contract_e2e.rs:393-407` — a **WALL** deadline, + `let deadline = Instant::now() + Duration::from_secs(30)`, panicking + `"the brain pulse must ingest the commune into the live context tier"`. **No elapsed.** + · `crates/spt-daemon/tests/sync.rs:98-110` — an **ITERATION** budget, `for _ in 0..400 { + net_streams(); sleep(25ms) }`, panicking `"sync stream never appeared at the responder's broker"`. + **No elapsed, no iteration count** — and its own comment calls it a "10s budget", which is a + NOMINAL sum of sleeps, not a wall bound: each `net_streams()` IPC call adds to it (hertz), so on a + loaded box the wall it grants is unknown and larger. +- **Four costs MEASURED in this one triage, each traceable to the missing clock:** + 1. **The panic reads as a CONTENT claim.** `contract_e2e.rs:403`'s message is what a 30 s DEADLINE + EXPIRY prints, and doyle relayed it to three peers as "a clean content assert, not a stall" + before opening the source. + 2. **History cannot supply margins at all.** nextest suppresses passing output by default, so + **eight consecutive goldens** of this cell's Windows durations exist and **not one carries an + internal margin**. (`--success-output immediate` retains it — so this is recoverable going + forward, and was not, going back.) + 3. **A whole-test total cannot locate the poll inside the test.** This conflation was committed + **five times in forty minutes by three different agents** — deployah twice (retracted both), + doyle once *in the message correcting deployah for it*, and it was caught each time by todlando + and hertz. A defect that costs three careful agents five errors in one morning is not a people + problem; the number they needed was not printed anywhere. + 4. **A FAIL and a PASS are not the same quantity.** `wait_for_stream_except` is called at + `sync.rs:201`, with `serve_sync`, the puller join and every assertion AFTER it — so a failing + run **truncates** there. This cell FAILED at 58.77 s and PASSED at 61.304 s, and that 2.5 s gap + carries no information in either direction. It was read as an anomaly ("it took LESS time and + still failed") until the call site was opened. +- **Remedy:** on expiry, print **elapsed, the budget, and iterations consumed**; where the poll waits + on another party, print that party's **PRODUCER PROGRESS** too — a watcher's own timings cannot + separate a slow watcher from a slow producer (hertz, todlando), which is the single fact this + triage most needed and could not get. Keep the content sentence; it is the panic's `{what}`, not + its whole body. +- **Population CENSUSED 2026-09-10 (doyle), discharging the census this entry declared OWED — 323 + poll sites detected, 304 of them classified blind. SUPERSEDED 2026-09-10 by the validation + below: 317 blind of a 321-site population, and the census's own coverage measured.** Read-only, run beside golden diagnostic 34458491560 + without dispatching anything to either runner. Preserved with its method, its raw classification + and its discarded first attempt at `.spt/preserved/ir95-poll-census-20260910/`. +- **Method, deliberately WORDING-INDEPENDENT.** Over the 212 tracked `*.rs` files under + `**/tests/**` plus `xtask/`, take every loop by SHAPE — `for _ in 0..N`, `while Instant::now() < + deadline`, `while .elapsed() < budget`, bare `loop {` — keep it only if it SLEEPS (a poll, not + a spin) and can FAIL (`panic!`/`assert!`/`expect(`/`bail!` on its exit path), then ask whether its + failure path emits a MEASURED VALUE (`.elapsed()`, `.as_millis()`, an iteration counter). **String + literals are stripped before that question is asked**, so the word "budget" inside a panic message + cannot credit itself with a clock. **Result: 323 sites, 19 apparently clocked, 304 blind.** +- **304 of 323 DETECTED CANDIDATES CLASSIFIED BLIND — a classifier count, NOT a proven floor.** + Stated at that strength on hertz's and todlando's correction of a stronger form this entry + briefly carried. What was actually spot-checked is the CLOCK bucket, and both errors found there + ran the same direction, undercounting blindness: `brain_read_deadline.rs:87` was credited from + elsewhere in the read window while its real failure path is `panic!("whole-carrier brain never + connected to {name}")`, carrying no clock at all; `er_briefing_presented_e2e.rs:310` returns + `Some(started.elapsed())` on SUCCESS and a bare `None` on expiry. +- **What that does NOT license.** Finding only false-CLOCK errors does not exclude false-BLIND ones, + and nothing here establishes POPULATION COVERAGE — whether the four loop shapes catch every poll + in the corpus is unvalidated, so a poll expressed as a helper, a retry combinator or an iterator + chain is uncounted and unmeasured. **No repository-wide `>= 94%` claim is made and none should be + repeated from this entry.** The honest sentence is: 304 classified BLIND among 323 detected + candidates, with the checked errors so far undercounting blindness. Validating both buckets and + the population is owed before any bound is asserted. +- **VALIDATED 2026-09-10 (doyle) — all three owed checks discharged; the headline figure MOVES to + 317 blind of 321.** Preserved as `VALIDATION.md` beside the census (machine-bound per [[IR-90]]). + (1) **The CLOCK bucket is now adjudicated EXHAUSTIVELY, not spot-checked**: of 19, only **4** emit + a measured value on their own expiry path, **13 were FALSE CLOCKs** and move to BLIND, and **2 are + not poll-failure sites at all**. Two mechanisms cover all 13, neither of them a wording problem: + **window bleed** (7 — four credited by a `const CONVERGE_BUDGET: Duration = …as_millis()…` + *declaration* 20-25 lines BELOW a bare `panic!("brain could not connect")`, one by a different + loop 57 lines away, two by an unrelated `now_ms()` helper) and **condition-use, not emission** (6 — + `assert!(started.elapsed() < Duration::from_secs(30), "…viewer_count={count:?}")` reads the clock + in the PREDICATE and omits it from the MESSAGE). Stripping string literals stopped a message + crediting itself; it does not stop CODE crediting a site whose message stays silent. + (2) **False-BLINDs checked at last — a seeded 30-site random sample of the BLIND bucket + (`random.seed(95)`), hand-adjudicated: 30 of 30 confirm BLIND, zero false-BLIND.** By the rule of + three that bounds the false-BLIND rate at **<= ~10%, at 95% confidence** — a binomial upper bound + assuming the 30 are a simple random sample of the 304 and each site's adjudication is independent; + it is **not a deterministic guarantee**, and zero errors in 30 excludes nothing outright (hertz). + This is the FIRST check run in the + direction that would REDUCE the blind count and it found nothing; every error found in either + bucket still runs the other way. + (3) **Population coverage, measured against every `sleep` in the corpus** — 472 call-sites, 349 + inside a detected window, 75 one-shot settles in no loop. The remaining 48 dedup to 47 loop + headers and **every one is explained, 0 unexplained**: **32** matched a census shape but cannot + FAIL within 60 lines (excluded BY DESIGN — the census counts loops that wait AND can fail), and + **15 are the detector's true false negatives**, a fixed nameable set: + `while std::time::Instant::now() < deadline` (the PATH-QUALIFIED spelling — the regex demands a + bare `Instant::now()`; `broker.rs:389`, `shellchan.rs:114`), `while !stop.load(Ordering::Relaxed)`, + `while (…)` (`shell_relink_force_e2e.rs:194/241`, `shell_stale_online_e2e.rs:146`, + `redispatch.rs:269`), and one `for (a, b) in [...]`. Those 15 stay UNCLASSIFIED. + **The 32 change how this census may be QUOTED:** "323 poll loops that wait and can fail" silently + excludes 32 loops that wait and CANNOT fail at the poll — a defensible scope, but a scope, and the + sentence must carry it. +- **Restated census, and the form to quote.** **321** sites in the adjudicated population (323 minus + 2 membership errors from the CLOCK bucket; a 3rd — `net_worker_starve.rs:99`, a fixed-window + SAMPLER that always runs its window and has no expiry failure — turned up in the BLIND sample, so + 3 errors in 49 hand-adjudicated sites means the population carries ~6% slop and **321 is not + exact**). **317** classified BLIND. **4** report a measurement. **15** poll loops outside the + detector's reach, unclassified. **Still not established, and still not to be written: that 317 is + a FLOOR, or any repository-wide rate.** One direction is now exhaustive, the other bounded at + <=10% by a 30-site sample — a bounded estimate, never a floor, and silent about the 15. +- **A distinction this validation surfaced, and the remedy above should carry: "emits a clock" is + TWO different things.** `transport_death_eof.rs:540` prints the OBSERVED quantity + (`{canary_after}ms`); `er_brief_once_per_session_e2e.rs:288` prints the MARGIN (`elapsed=`). Only + the second tells a reader how close the budget came. 3 of the 4 surviving CLOCK sites report a + margin, 1 reports an observation — and **a binary CLOCK/BLIND classifier cannot see that + difference**, which is a limit of the census, not of the sites. +- **One edit to preserved evidence, DECLARED.** `census.py` printed `<-- IR-95 population (a FLOOR)` + on its blind line. That sentence was retracted the day it was written, but the retraction never + reached the SCRIPT, so re-running the preserved artifact reprinted the withdrawn claim at whoever + ran it — a retraction does not propagate to the next reader on its own. The LABEL, and only the + label, is corrected in place; the computation is untouched and **still reproduces 323 / 19 / 304**, + which is what makes this validation checkable against it. The old line is quoted verbatim in + `VALIDATION.md`. +- **A FIRST detector was discarded, and its failure is itself evidence for cost 1 above.** Keying on + expiry VOCABULARY ("never", "timed out", "did not appear") found 53 sites and **missed + `contract_e2e.rs:393` entirely — the very site this entry was filed from** — because its panic + message is a CONTENT sentence. A wording-keyed census is structurally blind to exactly the sites + whose messages mislead. Recorded so no one re-runs the cheap version and reports 53 as the answer. +- **Positive control passes, and it found siblings.** Both originating sites appear, both BLIND: + `sync.rs:102` and `contract_e2e.rs:395`. The census also surfaced **two fixed budgets nobody had + noticed** — `sync.rs:89` (`0..300`, beside the `0..400` at :102) and `contract_e2e.rs:51` + (`0..400`, beside the wall deadline at :393). Concentration: `inject_control_wedge.rs` 18, + `broker.rs` 10, `endpoint_lifecycle.rs` 9, `dispatch.rs` 7, `twohost.rs` 7. +- **PRODUCER-PROGRESS INSTRUMENT — SPEC, 2026-09-10, and the three readings it does NOT support.** + The remedy above asks for the waited-on party's progress because a watcher's own timings cannot + separate a slow watcher from a slow producer. `last_streams_seen=0` throughout is equally + consistent with "the producer never produced" and "the watcher never saw it", and neither arm of + the releases#294 pair can tell those apart. **Design (doyle), narrowed by hertz and todlando, who + independently found the same three defects in the first form of it.** + **INSTRUMENT** — rides the IPC the poll ALREADY makes, so it adds no channel and no budget: the + responder's broker surfaces, alongside `net_streams`, a monotonic `streams_created_total` AND a + monotonic `streams_retired_total` for the session, the sequence and age of its last transition, + and the STREAM IDS of the most recent transitions so the watcher can correlate them against the + ids it sees and skips. The watcher prints the producer's counters beside its own observation. + **SEMANTICS THAT MUST BE DECLARED BEFORE A SINGLE READING IS TAKEN** — a bare count is not a + measurement until these are fixed: WHICH HOOK "created" fires at (broker stream-table + registration? wire acceptance?), WHICH KEY the counter is scoped to and what a session rotation + does to it, the COUNTER LIFETIME and reset behaviour (process lifetime, per session, what a broker + restart does), and WHICH CLOCK the age is measured against and whether it is comparable to the + watcher's. + **WHAT EACH READING LICENSES, and what it does NOT** (the first form of this table asserted three + causal conclusions it had not earned): + · `created=0` at expiry means **no creation was RECORDED AT THAT HOOK, in that counter's scope, + for that session key**. It does **NOT** establish that the requester never produced, and it does + **NOT** exonerate the watcher — the hook may sit downstream of the real production, or be scoped + to a different key. + · `created>0` with every snapshot empty is **CONSISTENT WITH a visibility/registration defect AND + with a stream being created and retired entirely between two 25 ms polls**. On its own it + establishes neither; the retire counter and the ids are what separate them. + · **Creation AFTER expiry cannot be observed by a poll that has already stopped.** A last-change + age cannot supply it. That needs either a separately bounded, separately declared post-expiry + observation, or a retained producer event trace with timestamps. + **Process constraints:** separately versioned and announced, never slipped between arms of a + comparison, and the existing diagnostic patch and every budget stay UNCHANGED. Widening a budget + to make a red go green remains refused. +- **`last_streams_seen` IS A FILTERED-VIEW COUNT — the sharpest narrowing this field has had, and it + applies to all three expiries in the releases#294 pair.** The poll calls `net_streams`; the broker + answers at `broker.rs:9546` `dispatch_net_streams` -> `host.dispatch_stream_infos()` + (`nethost.rs:1952`) -> `stream_infos_filtered(FALSE)`, and that filter drops TWO classes + SERVER-SIDE, before serialization (ADR-0040 decision 2, `REQ-STREAM-INFOS-SERVER-FILTER`): + `!e.retired.load(SeqCst)` and `!e.initiated_locally`. So `last_streams_seen=0` does **NOT** mean + the broker held no streams. It means **zero NON-RETIRED, NON-LOCALLY-INITIATED rows at that + instant** — a row that exists and is retired is invisible to the poll BY DESIGN. That is narrower + than the corrected form the RCA had settled on ("the final snapshot was empty"), and it is a + hook-scope fact of exactly the kind the preconditions above demand be pinned before any reading. +- **A "reuse it, no product change" stage-1 design WAS PROPOSED AND IS WITHDRAWN — recorded because + the reason generalises.** `NetHost::stream_counts()` (`nethost.rs:1980`) is already surfaced as + `NetStatusReply.streams_held` (`broker.rs:9002`, `msg.rs:874`) and documents itself as + "retirement-blind ON PURPOSE ... the resource truth the eligibility view hides". Reading it beside + `net_streams` looked like it would separate "filtered out" from "absent" for free. It does not, + for four reasons, and hertz and todlando raised them independently: + · **Two IPC replies are not an atomic snapshot.** `filtered=0` then `held>0` is equally consistent + with a row ARRIVING BETWEEN THE TWO CALLS. Such a pair must never be labelled proof of filtering, + and needs call order and timestamps recorded if it is read at all. + · **`streams_held` is HOST-SCOPED, not exchange-scoped.** It counts unrelated and locally-initiated + rows the filtered view drops, so it cannot speak to the exchange under investigation without + stream-ID correlation; any conclusion from it stays aggregate. + · **It adds UNMEASURED perturbation** — a second IPC per iteration changes the 400-iteration + window's wall duration by an amount nobody has measured. **CORRECTED 2026-09-10 (hertz, + todlando), because the first form of this bullet was itself an overclaim in two ways and both + are instructive:** (a) it called perturbation DISQUALIFYING, and it is not — instrumentation + that perturbs timing can still be useful; the objection is ADDED, UNMEASURED perturbation, not + perturbation as such. (b) It SIZED that perturbation from two figures that cannot carry it: the + measured `net_streams` cost (60,295 us / 400 = 0.151 ms per iteration) does **not** estimate the + `net-status` cost — different call, different work — and the **0.29% expiry difference between + the two arms is NOT an established causal effect size**, only an unexplained gap between two n=1 + observations. Sizing an observer effect against it treats an open question as a known + quantity. + · A single POST-EXPIRY sample avoids perturbing the poll but then describes LATER state, not + expiry state, and is still host-scoped. +- **THE INSTRUMENT THAT SURVIVES REVIEW: a retained, EXCHANGE-CORRELATED create/retire TRACE** + (hertz and todlando, converged independently). Recorded per stream lifecycle transition on the + producer side — stream id, conn id, remote id, transition, `initiated_locally`, a monotonic + sequence and a timestamp on a NAMED clock — retained and read AFTER the fact rather than sampled + by the watcher, so it adds **NO POLL CALLS**. Say that, and not "adds nothing" (hertz): it still + adds PRODUCER-SIDE work that may affect arrival timing and resource contention, and **that effect + is unmeasured too**. The trace is preferable for having no watcher IPC, not for being free. It captures between-poll transitions, + which neither a gauge nor an in-loop counter can: **a gauge reads 0 for a stream created and + retired between two 25 ms polls exactly as it does for one never created.** Correlation to the + exchange is by stream id against the ids the watcher saw and skipped. + **MEASURED: it does not exist today** — zero stream-lifetime log lines in `nethost.rs`. So this is + a PRODUCT change, not a reuse, and it stays unbuilt while the product is frozen. Its preconditions + are the ones above, unchanged. Production AFTER the trace's retention window remains outside it. +- **THE FIRST MARGINS THIS ENTRY PRODUCED UNDER CI / GOLDEN CONDITIONS — instrumented run + 34458491560, 2026-09-10 + (hertz's patch; deployah's lane; read independently by doyle).** The remedy above was argued for + on the ground that a poll which prints nothing cannot say how close it came. It has now printed + on a runner. **NOT the first margins overall** — the earlier local 12-leg run already reported + them, including a 392/400 (hertz, todlando). **The genuinely novel quantity is narrower and + sharper than "CI margins" (deployah): it is a FAILING poll's margin, which no local leg could + produce because all 12 of them PASSED.** A budget's behaviour at expiry is the one thing a green + run structurally cannot report, and it is the case the remedy exists for. + **The figure is the MAXIMUM AMONG SUCCESSFUL polls in this run — explicitly NOT a ceiling, and not + a bound on observations**: the poll that EXPIRED in this same run consumed 400/400 = 100%. So: + among the polls that PASSED on hfenduleam, both instrumented sites peak at ~63% of budget. The + seven successful + `[IR95-SYNC]` waits in that run are 92, 19, 72, 254, 21, 21 and 20 iterations — **worst 254/400 = + 63.5%** (6.449393 s) — and the `[IR95-CONTRACT]` success consumed **18.857089 s of its 30.000 s + wall = 62.9%**. Two independent sites, two different budget KINDS, the same maximum-among-passes. + Headroom at that worst SUCCESSFUL case ~37%; the failing case had none left. **n=7 and n=1, one + box, one run: a first CI measurement, not a stability claim, and NO budget change is proposed on + it.** +- **The iteration budget's WALL EQUIVALENT, also measured for the first time.** The expiring poll's + 400 iterations cost **10.199907 s against 400 × 25 ms = 10.000 s nominal — 2.0% over**, with + per-iteration overhead ~0.5 ms against a 25 ms sleep (IPC 60.295 ms total, 0.151 ms per iteration, + **0.59% of elapsed**; sleep is 99.4%). So on this box in this run a "400 iteration" budget and a + "~10.2 s wall" are nearly the same object — the iteration budget did NOT silently become a + different budget under load. That bounds one standing worry about blind iteration counts, and it + is precisely the class of thing that stops being guessable the moment a clock prints. Box-local + and run-local; it is not a general result. +- **A scope limit the instrumentation itself imposes, and a smaller ask than it first looks.** + `last_streams_seen` is an ASSIGNMENT per iteration (`sync.rs:111`), not an accumulator, so + `last_streams_seen=0` at expiry describes the FINAL iteration only — not all 400 (todlando's + correction; deployah withdrew the aggregate reading, doyle confirmed it at source). But the loop's + CONTROL FLOW already proves the interesting half for free: the `find()` at `:113` returns the + instant a non-skipped stream appears, so **reaching the expiry print IS proof that no non-skipped + stream was seen in any of the 400 SNAPSHOTS** — no aggregate trace required. **It does NOT mean + no stream EXISTED between snapshots** (hertz): the poll samples at 25 ms, and nothing here + observes the gaps. What is genuinely + unmeasured is only whether a SKIPPED stream was ever present, and `skip` is empty on that call. + An aggregate/per-iteration trace therefore buys less here than the gap first suggests. +- **NOT ESTABLISHED: that all 304 deserve a clock.** The census SIZES the problem; it does not + schedule it, and it does not pre-judge which sites are worth the edit. Remedy triage is owed + separately and is the larger half. +- **Kin:** [[IR-84]] (the one `PUMP_PEER_FAIL` arm the rigs fire was the only one carrying no clock — + **the same class in product code**; this entry is its test-corpus face), [[IR-85]] (budgets sized + against a box whose speed moved), [[IR-77]] (nothing stamps the spawn-to-first-log interval, so a + readiness red cannot say where the time went), [[IR-90]]. +- **Ripe when:** with the releases#294 instrumented base-vs-head comparison, which **cannot be run + without it** — the experiment's whole output is the margins these sites do not print. · + **Size:** small per site; the census is the larger half. + +### IR-96 — the `daemon_stop_convoy_e2e` API-storm cell fails 3-of-3 under the HEAVY suite at the diagnostic tree while passing at the candidate in golden CI, and nothing yet separates instrumentation from box from product + +- **Status:** OPEN, **cause-unclassified** (todlando's and hertz's stipulation, adopted verbatim — a + triage placement here is NOT a finding of non-defect). +- **Origin:** releases#294 / v0.69.0 Arm B, 2026-09-10, deployah executing at diagnostic tree + `ed8ecd94` (= candidate `a9e786b2` + the unlanded sync instrumentation, never merged). +- **What:** `spt::daemon_stop_convoy_e2e stop_under_an_api_storm_stays_down_then_start_brings_up_exactly_one` + FAILED in **all three** Arm B runs, with durations **growing every run: 9.473s · 10.973s · + 14.553s**. Different signature from the #294 sync cell; nothing was filed against #294. +- **THE DIFFERENTIAL that makes it worth an entry rather than a shrug:** the same cell shows **0 + failures at the candidate** in golden 34481993681 Phase A on this same box — that run's single + failure was the sync cell (`3381 tests run: 3380 passed (6 slow, 5 leaky), 1 failed, 1 skipped`). + So: 0-of-1 at `a9e786b2` under CI load, 3-of-3 at `ed8ecd94` under local HEAVY load. +- **WHAT THAT DOES NOT ESTABLISH.** It does **not** isolate the instrumentation or the box as the + only candidates — a condition-dependent or intermittent PRODUCT defect remains live, and a single + CI pass is not evidence of absence. doyle claimed that elimination on 2026-09-10 and **withdrew + it** the same hour under todlando's and hertz's correction; the withdrawal is recorded in + `.spt/preserved/gate-r2-34481993681-20260910/CLASSIFICATION-AMENDMENT-2-doyle.md`. +- **Confounded by IR-97** (below): the three runs were not equivalent conditions. +- **Ripe when:** any lane touches daemon stop/start convoy behaviour, OR a golden run reds this cell + at a candidate, OR someone runs the cell at `a9e786b2` under the same local HEAVY workload — that + last one is the cheap discriminator and nobody has run it. · **Size:** one targeted local run to + discriminate; unknown beyond that. + +### IR-97 — a suite that leaks a fixed cohort of child processes per invocation makes consecutive runs non-comparable, and the between-run accounting the protocol required was not enforced + +- **Status:** OPEN. **Protocol deviation** (hertz's framing, and it is stronger than "confounder": + the protocol ALREADY required between-run accounting — execution did not enforce it, so the fix is + enforcement, not a limitations paragraph). +- **Origin:** releases#294 Arm B, 2026-09-10. Derived by doyle from deployah's preserved + `10-residual-births.json` against the run windows. +- **What:** each Arm B suite leaked **exactly 6** `spt` children that outlived it; all **18** were + still alive at the post-B3 census. Because a process alive at census whose OS birth time falls in + an earlier window existed continuously through the runs between (pid + birth time is an identity), + the carried diag-tree residue was **0 during B1 · 6 during B2 · 12 during B3**. The three + "repetitions" were therefore not three equivalent conditions, and every cross-run comparison in + that arm inherits it. +- **LIMITS, stated so the entry is not over-read:** process **COUNT** is measured; **LOAD is not** — + idle orphans may cost nothing, contention was UNMEASURED (overlap UNVERIFIED all arm), and this is + **not** a drift meter. It neither implicates nor excludes residue as a cause of the one expiry, and + it does not explain it: B3 carried twice the orphans and **passed**. +- **Also unestablished, in both directions:** 18 censused against 15 killed shows only that three + were absent at the second read. "Three self-exited" needs exit evidence; "I killed 18" was + likewise refused by deployah at the time. +- **Kin:** the LEAK rows every Arm A/B run reported; `REQ-HAZARD-RESUME-CUSTODY-ABA` (KH 7.51) and + KH 7.58 — the cleanup that resolved this was correctly scoped by **pid + image path**, leaving 15 + unrelated `spt` processes untouched, where a name-based kill would have taken down the live agent + fleet including the gater's own session. +- **Ripe when:** any repeated-run experiment is designed on this box — census BETWEEN runs, not only + at the end. · **Size:** a between-run census step plus a scoped reap in the harness. diff --git a/docs/KNOWN-HAZARDS.md b/docs/KNOWN-HAZARDS.md index 7043eec8..039c2456 100644 --- a/docs/KNOWN-HAZARDS.md +++ b/docs/KNOWN-HAZARDS.md @@ -1033,6 +1033,20 @@ The kill-path rule above generalizes: `daemon.pid` is not authority for *"which --- +### 7.66 A bounded retry wait that gates an assertion records its MARGIN on success and declares its expiry CENSORED on failure — a pass/fail loop is not a meter `[REQ-HAZARD-RETRY-MARGIN-UNMEASURED]` + + +- **Failure (paid-for, releases#294 / v0.69.0: gate runs `34445961595` and `34481993681`, two reds at the SAME cell, `spt-daemon::sync two_tier_sync_lands_and_gate_refuses_server_side`):** the wait that gates the assertion is the house-standard 400 iterations x 25 ms — a 10.0 s budget — and it reports PASS or FAIL and nothing else. The elapsed is discarded. Two full gate arms across two boxes were spent and the cause of the red is still **UNCLASSIFIED**, because the instrument could not answer a question in either direction. +- **Defect (a) — a green carries no margin.** Instrumented at 400 observations on the Windows cell, successes landed at **4.448 s** (observation 176/400) and **9.940 s** (observation 392/400). Those are the SAME GREEN to the loop. A pass one scheduling hiccup from red is indistinguishable from a healthy one, so **a history of passes is not evidence of headroom** — every prior green of this cell is silent about margin, including the greens on the record head. +- **Defect (b) — an expiry is RIGHT-CENSORED, not a sample.** The loop stops looking at the bound (measured 10.156–10.200 s wall), so it establishes that no eligible condition was observed BEFORE expiry and **nothing whatsoever about after**. A censored expiry has **no upper endpoint**: it cannot be averaged in, cannot be called a latency or "the tail", and cannot separate DELAYED from ABSENT, RETIRED, or FILTERED. Write `CENSORED at X, no upper endpoint`. +- **THIS IS THE SEAM THAT WITHDREW THREE CAUSE PHRASINGS IN ONE HOUR** (2026-09-10, all mine): *"delayed publication, measured"* — censoring, caught by hertz; *"a ceiling outside the measured tail"* — unspecifiable, hertz **refused to implement rather than invent a number**, and that refusal was worth more than compliance; *"boundary defect MEASURED"* — caught by todlando, **proximity to a threshold is a RISK SIGNAL, never a verdict that the threshold is wrong**. If the 9.940 s success is itself pathological, the budget is innocent. An unmeasured loop cannot tell those apart, which is exactly why the budget was **not** changed. +- **Invariant:** (1) a gating retry wait records **elapsed-at-success against its budget** and surfaces it on the reporting path — never a bare `passed`; (2) an expiry is reported as **CENSORED AT ``, NO UPPER ENDPOINT**, never as a sample; (3) where the eventual outcome matters, expiry starts a **separately bounded diagnostic continuation that PRESERVES the original failure** — a continuation never converts the original failure into a pass — reporting first-observed elapsed, or an **INCOMPLETE OBSERVATION** rather than an absence if it too is bounded out; (4) an interval is named for **what was actually hooked** — *submission-to-registration* is not *wire latency*, and a window spanning process spawns and filesystem work is not a *publication* meter. +- **Why the naming clause is part of the hazard, not an aside:** the waited window at this site is COMPOSITE by construction — `request_sync_on` opens and subscribes, then runs `git rev-parse` once per requested ref BEFORE `net_stream_send`, and a first pull requests four refs while creating the requester's fresh bare Git store inside the timed overlap; QUIC's `accept_bi` becomes visible only on the peer's first write (`nethost.rs:1071-1089`). Calling that number a publication latency — as I did — attributes a composite of git init, four subprocesses, a first write and a visibility gap to one of its parts. +- **spt-core mapping:** measured site `crates/spt-daemon/tests/sync.rs:109`. **SCOPE GUARD — the 400x25 ms shape is house-wide** (`access.rs:80`, `attach.rs:114`, `broker.rs:618`, ~20 more): a lane fixes **only the measured site** and says so in the commit. A blanket rewrite of every call site is a separate, argued change, not a rider. +- **Source:** releases#294 gate triage (doyle ruled; hertz instrument design — the 400-iteration acceptance boundary kept, continuation added, **upheld over my wider-ceiling version**; todlando source-read mechanism candidate and the proximity correction; deployah `.spt/DIAG-ARM-CRITERIA-294.md`). Kin 7.61 (a monitor's green producible by the fault it watches — same family: a green that is not evidence). + +--- + ## Conformance checklist (condensed) | # | Invariant | spt-core surface | diff --git a/traceable-reqs.toml b/traceable-reqs.toml index 20a64909..33f27d86 100644 --- a/traceable-reqs.toml +++ b/traceable-reqs.toml @@ -7624,3 +7624,8 @@ required_stages = ["doc", "impl", "unit", "int"] # ACTIVATED WEBSERVE W3 (todla id = "REQ-WAN-REPLY-BOUND" title = "A CROSS-NODE REQUEST VERB BOUNDS ITS REPLY READ AND REPORTS A SILENT PEER AS ITS OWN OUTCOME, NEVER AS NoReply (releases#289, doyle ruled 2026-09-09). The six one-shot request verbs in spt-daemon wan.rs -- request_wan, request_fork, request_redeem, request_answer, request_knock, request_presence -- send one record, finish their side, subscribe, then read in an UNBOUNDED loop whose only non-error exits are a decoded reply and a stream EOF. A peer that ACCEPTS the stream and never replies and never finishes it wedges the caller forever; pre-existing since ec360f16, not a WEBSERVE regression. THE FIX IS THE CARRIER, NOT A BUDGET AT THE CALL SITE, and that distinction is the requirement: a deadline handed to a blocking Whole carrier is REFUSED outright (ErrorKind Unsupported, brain.rs) because a timeout there would abandon a half-read frame and desync the conn, so the six verbs must run on the pump-mode Split carrier built by cold_start_pump, exactly as the cross-node digest pull already does. THE SEAM ENFORCES IT rather than documenting it: one shared refuse_unbounded_carrier helper, called before any wire I O in all six verbs, refuses an unbounded carrier by name and names cold_start_pump as the remedy -- the two carriers are indistinguishable at the call site (both are a mutable Brain reference), and a comment in exactly this position already failed to stop one wrong wiring. A BUDGET EXPIRY IS A DISTINCT OUTCOME: NoReply already means the receiver FINISHED the stream without a reply frame (an old or refusing receiver), so reporting silence as NoReply would report a peer that said nothing as a peer that answered nothing -- different facts, different remedies. The expiry rides the brain own reply_read_deadline so one wire has one notion of a peer gone quiet, and it is read through read_event_until rather than read_peer_reply_until because the latter reclassifies the timeout kind away for the pump per-peer drop arm. Gate: impl -- the shared refusal helper, the six carrier constructions in the CLI send paths, the bounded loops, and the distinct outcome threaded through every report surface so a timeout never prints as sent; unit -- the refusal names cold_start_pump, and the distinct outcome is asserted DISTINCT from NoReply and from the happy token in each report classifier; int -- a real dialable peer that accepts the stream and never answers returns the distinct outcome inside the budget rather than hanging, with the caller bound asserted rather than merely survived." required_stages = ["doc", "impl", "unit", "int"] # ACTIVATED releases#289 (todlando lane start 2026-09-09, doyle ruled the carrier shape and the two-commit scope the same day). impl = the shared refuse_unbounded_carrier helper + cold_start_pump at the six wansend call sites + the bounded loops + the distinct expiry variant threaded through classify_wan_reply and its siblings. unit = the refusal message names cold_start_pump; each classifier maps the expiry to its own arm, asserted distinct from NoReply. int = the silent-peer cell (a real broker that never serves the family) on the digest_cross_node.rs precedent. Kin REQ-HAZARD-PUMP-IPC-DEADLINE (the refusal this reuses) and REQ-DIGEST-CROSS-NODE-PULL (the same fix, one release earlier, on the digest family). doc = the WAN_PEER_SILENT row in the messaging overview failure-verdict table, which is a CLOSED set adapters build blind from -- a new failure tag with no published row is a silent contract change (doyle ruled 2026-09-09, commit 1b). + +[[requirements]] +id = "REQ-HAZARD-RETRY-MARGIN-UNMEASURED" +title = "releases#294 / v0.69.0 gate runs 34445961595 and 34481993681 (doyle 2026-09-10): a BOUNDED RETRY WAIT that gates an assertion must record its MARGIN on SUCCESS and declare its expiry CENSORED on failure -- a loop that reports only pass/fail is not a meter and cannot be reasoned from in either direction. SHAPE: the house-standard 400 iterations x 25 ms (10.0 s budget) at crates/spt-daemon/tests/sync.rs and roughly twenty sibling sites (access.rs:80, attach.rs:114, broker.rs:618, others) polls a condition and asserts on the outcome, discarding the elapsed. TWO DEFECTS, OPPOSITE DIRECTIONS. (a) ON SUCCESS the elapsed is thrown away, so a 4.448 s pass and a 9.940 s pass are THE SAME GREEN -- both measured on the same Windows cell in one 400-observation instrumented run, 176/400 and 392/400 -- and a green one scheduling hiccup from red is indistinguishable from a healthy one. Every prior green of this cell is therefore silent about margin, so a history of passes is NOT evidence of headroom. (b) ON FAILURE the observation is RIGHT-CENSORED: the loop stops looking at expiry (measured 10.156-10.200 s wall), so it reports that no eligible condition was observed BEFORE expiry and NOTHING about after. A censored expiry has NO UPPER ENDPOINT: it cannot be averaged into a tail, cannot be called a latency, and cannot separate DELAYED from ABSENT, RETIRED, or FILTERED. THREE CAUSE PHRASINGS WERE WITHDRAWN IN ONE HOUR on exactly these grounds -- 'delayed publication, measured' (censoring, hertz), 'a ceiling outside the measured tail' (unspecifiable endpoint, hertz refused to invent a number), and 'boundary defect MEASURED' (proximity to a threshold is a RISK SIGNAL, not a verdict that the threshold is wrong -- todlando). PROXIMITY IS NOT A VERDICT: if the 9.940 s success is itself pathological then the budget is innocent, and an unmeasured loop cannot tell those apart, which is why the budget was NOT changed. INVARIANT: (1) a gating retry wait records elapsed-at-success against its budget and surfaces it in the failure/report path, never a bare 'passed'; (2) an expiry is reported as CENSORED AT , NO UPPER ENDPOINT, never as a sample or a tail; (3) where the eventual outcome matters, expiry starts a SEPARATELY BOUNDED diagnostic continuation that PRESERVES the original failure -- a continuation never converts the original failure into a pass -- and reports first-observed elapsed, or reports an INCOMPLETE OBSERVATION rather than an absence if it too is bounded out; (4) an interval is named for what was actually hooked -- submission-to-registration is not WIRE LATENCY, and a composite window spanning process spawns and filesystem work is not a publication meter. SCOPE GUARD: the 400x25 ms shape is house-wide, so a lane fixes only the measured site and says so in the commit; a blanket rewrite of ~20 call sites is a separate, argued change." +required_stages = [] # NOT ACTIVATED -- doyle 2026-09-10, registry-first per the activate-don't-pre-fail rule. Activated by whichever lane delivers the test: doc = KNOWN-HAZARDS.md 7.66 (written with this row), impl = margin recording + censored-expiry reporting at the measured site (crates/spt-daemon/tests/sync.rs), unit = a success reports its elapsed and an expiry reports CENSORED with no upper endpoint. hertz's diag/294-sync-stages continuation is a DIAGNOSTIC ARM, not a landing, and mints no stage here.