# Flake ledger

Tests that have failed nondeterministically, with mechanism notes and hardening
status. Discipline: **any flake gets an entry, not a shrug** — a second
occurrence upgrades the entry from "observed" to "harden it." Rig flakes from
the two-host ladder (docs/TWO-HOST-RUNBOOK.md) land here too.

<!-- [doc->REQ-FLAKE-LEDGER-KEYED-ROWS] -->
**Rows are keyed by the test (or CI step) name, never by position.** There is no
row number: a hand-numbered first column made two lanes appending "row N+1"
conflict textually AND mint the same number, and a forward-referenced number can
be squatted by a later, unrelated row — that is exactly how every in-tree
`FLAKE-LEDGER #15` citation came to point at the wrong entry, and how the
daemon-tree bring-up row below went a year unwritten while 20 sites cited it.
Cite an entry from elsewhere in the tree as `FLAKE-LEDGER: <test name>`.

Append new sightings at the end. A BACKFILL of a historical flake goes in date
order instead, and says in its own status column that it is reconstructed and
what evidence did not survive — a row that cannot be told apart from a
contemporaneous one is a row that will be cited as if it were.

| test | occurrences | mechanism | status |
|------|-------------|-----------|--------|
| `spt-daemon brain_restart_survives_gaplessly` | 1× hosted Windows (2026-06-03, hung) | ConPTY stall under hosted-runner load | job `timeout-minutes` bounds it; not seen self-hosted |
| `spt-term digest_parses_a_real_pty_stream` | 3× gravity (2026-06-03; 2026-06-04 runs 26957386850, 26966738231) | sprint-collapse assert (`digest.rs`) — PTY input echo and the child's write are **concurrent writers** into the master-read stream; back-to-back `send_line`s let a later line's echo interleave mid-`Write(…)`, breaking the tool regex for that turn | HARDENED (2026-06-04): sends serialized — each line pumped back (adaptive echo-copy count, 2s/line bound) before the next, so no concurrent writer races an earlier line; failure asserts now print the raw transcript |
| `spt-daemon sync.rs torn_pull` + `concurrent_writes` | 1× (2026-06-03) | `wait_for_stream` deadline too tight under load | HARDENED: `wait_for_stream_except` 2s→10s; no recurrence |
| `spt-live context::write_context_suppresses_stale_llm` | 1× gravity (2026-06-04 run 26967808321) | test calls `write_context` (reads SPT_HOME node identity + epoch stamp) WITHOUT the `with_home` lock — a parallel `with_home` test's env swap + temp-dir teardown races the stamp's epoch write into a deleted home (`NotFound`) | HARDENED (2026-06-04): test wrapped in `with_home` — serializes on the home lock like every other SPT_HOME-touching test |
| `spt-daemon attach.rs:600` re-serve (`output gap: got seq 1 want 0`) | 1× kitsubito Linux (2026-06-15 run 27545857429, PR #14 stale-base CI) | re-serve replays the buffered output stream on re-attach; under load the new subscriber observed `seq 1` before `seq 0` — an ordering/timing gap in the re-serve sequence assert, not a content error. Subsystem disjoint from the triggering change (PR #14 = digest-proof CLI key-fill, cannot touch attach sequencing) | OBSERVED 1×: did NOT reproduce on the clean-base rerun (`gh pr update-branch` onto post-#15 main → green both runners). Not hardened; a 2nd occurrence upgrades to harden |
| `spt-term tests/stream.rs:49 bounded_backpressure_stalls_an_unconsumed_reader` | 1× hfenduleam Windows (2026-06-15 run 27545857429, PR #14 stale-base CI) | asserts an unconsumed bounded stream stalls the writer-side reader; got `16 -> 82` (reader advanced past the bound) — a timing-sensitive backpressure assert racing the bound check under hosted-runner load. Subsystem disjoint from PR #14 (digest-proof CLI) | OBSERVED 1×: did NOT reproduce on the clean-base rerun (green both runners). Not hardened; a 2nd occurrence upgrades to harden |
| `spt-daemon::attach attach_survives_target_brain_restart_exactly_once` | 1× kitsubito (2026-06-16 run 27595180782 attempt 1, M11-W3) | **TIMEOUT, not an assertion fail** — SLOW>60>120>180>TERMINATING, nextest TIMEOUT at the 240s cap (`attach.rs:600`). A heavy multi-process brain-restart-exactly-once test on the slow shared kitsubito box under concurrent load (n1-gate on the same runner + a parallel `ci.yml` docs-publish run firing simultaneously). W3-INDEPENDENT: Windows green end-to-end; ALL tunnel tests passed fast on Linux too (tunnel_e2e 0.941s, loopback_tunnel_backpressures_without_loss 1.091s) — a logic bug fails both runners deterministically, this timed out on one under load | observed; cleared by `gh run rerun --failed` (builds reused, no re-tag). Harden if it recurs — candidate: raise this test's per-test nextest timeout, or serialize it off the concurrent n1-gate slot |
| CI build step `Build notify-shell` (kitsubito Linux) | 1× kitsubito (2026-06-16 run 27652755792 attempt 1, v0.8.1 PR #17) | crates.io dep-download blip during the notify-shell build — `download of config.json failed, curl failed` fetching `serde_json`: a transient registry/network fetch failure on the runner, not a code or test fault (a real dep break fails both runners deterministically; this hit one runner once) | observed; cleared by `gh run rerun --failed` (attempt 2 green, builds reused). Harden if it recurs — candidate: a cargo fetch retry / registry cache warm on the runner |
| `spt dummy_harness_e2e` (BOTH tests: `endpoint_run_brings_up_a_long_lived_dummy_harness_and_rc_attaches` + `endpoint_run_attach_awaits_online_before_attaching`) | 1× kitsubito (2026-07-05, REMOTE-TRUTH Batch-2 re-gate @f2a799e; roles swap run-to-run) | **process-global `set_var` race in a multi-test e2e binary.** Both tests `std::env::set_var("SPT_HOME", own_tempdir)` then do IN-PROC reads (`perch::spt_home`/`adapters_dir`, `registry::register`, psychebin `fs::copy` staging). One file = one test binary → the two run on parallel threads; whichever set_var last wins for BOTH → `register`/staging land in the wrong home (`ENDPOINT_RUN_ADAPTER_UNREGISTERED`; os-32 sharing collision). Child procs were safe (explicit `.env`) — only the in-proc reads leaked. File untouched since F-030; Batch-2 broker-reap timing shift woke it. `--test-threads=1` → 2/2 green (proves serialize) | HARDENED (2026-07-05): file-local `static E2E_LOCK: Mutex<()>` held whole-body in each test (contract_e2e.rs:26 pattern); green at default parallelism. **CLASS = process-global `set_var("SPT_HOME")` + in-proc read in a multi-`#[test]` e2e binary.** doyle's gate2 (`--no-fail-fast`) proved it beyond dummy_harness: `live_adapt_translation_swap_e2e` (3/4 red) + `multi_subnet_bringup_e2e` (3/3 red), same signature. **SWEPT (anchored `grep -cE '^#\[test\]'` census):** locked `dummy_harness_e2e` (2), `live_adapt_translation_swap_e2e` (4), `multi_subnet_bringup_e2e` (3), `translate_proof` (2); `contract_e2e` (26) + `gateway_e2e` (35) already locked. EXCLUDED `brain_survive` + `n1_pairing` — each a SINGLE `#[test]` (no intra-binary parallelism → cannot race); both already carry an in-file "a 2nd `#[test]` must bring the env-lock" deferred contract (a never-contended Mutex contradicting its own doc is worse than nothing). NB: doyle's first census overcounted these two — an UNANCHORED `grep -c '#\[test\]'` counted the attribute mentioned in their comment prose; anchor the count |
| `spt-store registry::tests::concurrent_registration_never_locks` | 1× hfenduleam local gate (2026-07-05, REMOTE-TRUTH Batch-2 bless matrix @3f6f327, fresh worktree, `nextest --workspace --no-fail-fast`) | 16-concurrent-writer SQLite registration gate FAILED at 5.63s — the SQLITE_BUSY load-contention class ALREADY documented at `db.rs:60`: box saturated by the 1588-test full-parallel suite meant a writer couldn't schedule its retry inside the default 5s `registry_busy_timeout`. Isolated ×5: green, 0.25–0.35s each. NOT a product defect — the exact scenario the `SPT_REGISTRY_BUSY_TIMEOUT_MS` env knob (default 5s) was added for | observed 1× (local gate, not CI); in-code remedy pre-exists: set `SPT_REGISTRY_BUSY_TIMEOUT_MS=30000` on saturated gate rigs. Harden candidate on recurrence: bake the raised knob into gate-matrix/CI test invocations |
| `spt-daemon::resume resume_mode_brain_spawning_new_sessions_delivers_each` | 1× kitsubito (2026-07-05 run 28768663321 attempt 1, REMOTE-TRUTH [twohost] PR #52) | Phase-B heavy-class timing EOF — "session exited before the expected output arrived" (UnexpectedEof) at 62s under the serialized heavy pool. SAME FAMILY as this file's prior Linux-CI flake (7940318, D4-2b: sequential spawn/read hardening) — a spawned session's exit raced the expected output read on the slow shared box. File untouched by the wave; Windows leg green same run; the wave's spt-daemon changes (C1 broker lifecycle) are cross-platform and every local Win gate was green — but note this run was the wave's FIRST full Linux suite | observed 1×: cleared by `gh run rerun --failed` (attempt 2 green, builds reused, no re-tag — the `attach_survives_target_brain_restart_exactly_once` / `Build notify-shell` rerun-clears-it pattern). Harden if it recurs — candidate: extend the 7940318 sequential-spawn/read pattern to this test's spawn loop, or a per-test nextest timeout raise |
| `spt-daemon::input_ack_deadlock input_flood_through_serve_attach_does_not_deadlock_broker` | 1× hfenduleam local gate (2026-07-06, REMOTE-TRUTH F-2 matrix @a21bc6b, `nextest --workspace --no-fail-fast`) | deadline-margin under full-suite load: the flood exchange's `recv_timeout(30s)` (input_ack_deadlock.rs:512) blew under the 1608-test parallel pool + 7 leaky daemons — FAIL at 34.5s vs ~32s natural runtime isolated (the margin is ~3s by design). Subsystem disjoint from the triggering commit (F-2 = api/reporting.rs soft-end guard; cannot touch broker input-ack). Isolated ×5: green, 31.9–32.2s | observed 1× (local gate, not CI); harden candidate on recurrence: widen the :512 recv bound or serialize the flood test into the Phase-B heavy class (CI already runs it serialized — this class only bites full-parallel local matrices) |
| `spt::bin cli::tests::probe_all_cap_batches_into_windows` | 1× hfenduleam local gate (2026-07-06, stack matrix @0f08fa9 leg-3 rerun) | fast unit (0.52s natural) FAILED at 1.7s under the 1614-test full-parallel pool — probe-batching window assert with a timing component starved under load; subsystem disjoint from the triggering stack (F-2 reporting / E-2 grid title / picker rider). Isolated ×5: green, 0.52–0.55s. Same night as `input_flood_through_serve_attach_does_not_deadlock_broker` — the full-parallel local matrix under a busy box is the common factor, not the tests | observed 1× (local gate, not CI); no harden yet. NIGHT-PATTERN NOTE: `concurrent_registration_never_locks`, `input_flood_through_serve_attach_does_not_deadlock_broker` and this entry are all first-sightings from the same gate recipe (full-parallel nextest on a loaded shared box) — if a FOURTH distinct test trips this way, harden the RECIPE (nextest profile with heavy-class serialization locally, mirroring CI's Phase split) rather than the tests |
| `spt::bin rc::tests::attach_viewport_reconnects_across_a_broker_bounce` | 1× hfenduleam local gate (2026-07-06, WORKER-TRUTH W-3 build @worker-truth, `nextest -p spt --bin spt` full-parallel 360-test run) | **TIMEOUT, not an assertion fail** — SLOW>60…>TERMINATING at the 240s cap under the 360-way `--bin spt` pool. A REAL-broker + broker-bounce reconnect UNIT test (rc.rs:2071) that escaped the HEAVY-class serialization: the Phase-A/B split + `heavy-broker-pty` test-group target integration test BINARIES (`kind(test)`), but this is a `kind(bin)` unit test inside the `spt` binary, so it ran in the LIGHT pool at full parallelism and starved. Isolated ×1: green ~1.3s. Subsystem (rc attach/broker-bounce) disjoint from the W-3 worker-reap wave that triggered the run | **THE FOURTH DISTINCT FULL-PARALLEL TRIP** the NIGHT-PATTERN note on `probe_all_cap_batches_into_windows` called for — so HARDENED at the RECIPE, not the test: the rc real-broker/bounce unit class is folded into the `heavy-broker-pty` nextest test-group (`.config/nextest.toml`, a `kind(bin) & test(/^rc::tests::(…)$/)` override) so it serializes with the rest of the heavy class and can never be starved by the light pool again. EXTEND the name set on each new rc-broker sighting |
| `spt` daemon-tree e2e bring-up, PRECONDITION "brain never came up" — rotating across `bind_honest_cross_perch_e2e`, `live_adapt_translation_swap_e2e`, `multi_subnet_bringup_e2e`, `run_no_dup_session_e2e` | 6× kitsubito Linux (2026-07-15, across the FORKENING W3 + W4 gates — two independent runs), rotating membership run to run | **RECONSTRUCTED ENTRY — see the provenance note in the status column; this row was never written at the time.** Four `spt` e2e binaries that spawn a REAL `spt daemon run` tree (`CARGO_BIN_EXE` daemon run + ~10s bringup) sat in the LIGHT Phase-A pool, so a 16-way daemon storm on kitsubito blew the 30s brain-readiness deadline probabilistically. ROTATING membership across the two gates is the load-flake tell (a logic bug picks the same victim); `run_no_dup_session` SOLO on the warm tree PASSED in 10.7s. Exonerated by absence of the competing causes: no OOM / fork / TasksMax events on the box. Noted but not charged: v6-first DNS with no default v6 route, the broken-IPv6-iroh tell, recorded as a boot pad rather than this mechanism. All four QUALIFIED as HEAVY under `.config/nextest.toml`'s own sweep criterion and postdated the last sweep | **HARDENED AT THE RECIPE (2026-07-15, `5199464`)** — the four binaries added alphabetically to all three `<HEAVY>` strings (ci.yml Phase A exclude, Phase B include, nextest.toml `heavy-broker-pty` override; verified byte-identical), plus `daemon_refresh_e2e`, new on that branch, classed heavy AT BIRTH rather than after its first Phase-A flake. `composite_e2e` stayed LIGHT (daemonless apply, no daemon tree); `translate_proof` stayed Phase A by ruling (one sighting, single-child spawn, watching). **This row is the origin of the HEAVY-AT-BIRTH ruling** — classify a real-daemon-tree/real-broker test at file creation, not after its first Phase-A timeout — whose canonical text is the `<HEAVY>` stanza in `.config/nextest.toml`. **PROVENANCE (reconstructed 2026-08-02, releases#95, hertz; doyle-ruled after confirming the primary RCA log is no longer retained):** `5199464`'s commit message ends "Ledger entry appended." and its diff touches only `.github/workflows/ci.yml` and `.config/nextest.toml` — the entry was never written, while the citation `FLAKE-LEDGER #15` it minted spread to 20 sites and was later squatted by the unrelated servicehost row below (`e1a3338`, 2026-08-01). Reconstructed from that commit message plus the surviving citation sites; no run ids, no per-sighting timestamps and no raw gate logs survive, so this row carries the mechanism and the remedy but NOT the primary evidence, and must not be cited as if it did |
| `spt-daemon::servicehost_supervision_e2e bits_swap_under_the_hold_and_the_new_ones_come_up_on_release` | 1× hfenduleam Windows golden (2026-08-01 run 30720655534, sha 13e94d5) | **TEST DEFECT, not a load flake — the only entry here whose mechanism is the assert itself.** The new-process leg asserted identity by PID INEQUALITY (`assert_ne!(second, Some(first))`); Windows recycles pids aggressively enough under suite churn to hand the SAME pid back to the replacement, and the run failed with both sides `Some(47100)` while the log SELF-PROVED the swap (SERVICE_STARTED 47100 → SERVICE_QUIESCED cooperative exit under hold → SERVICE_STARTED 47100). The immediately-preceding asserts — `ServiceOutcome::Started` on release and the swapped-in binary's beacon — had already passed, so the functional swap was never in doubt: a pid was standing in for an identity it cannot carry past any exit (the KNOWN-HAZARDS `pid_started_at` class). File untouched by DOORBELL (last touch 1488e39, already on main) — pre-existing surface | **HARDENED (2026-08-01, releases#93):** the leg now compares the original's BIRTH STAMP (`proc::process_identity`, the `(pid, start-time)` pair) captured while it is alive against the same pid re-read after the ceremony — `Absent` when nothing took the pid, a DIFFERENT `Present` when something did, and only the original still running reads equal. Guarded by an INSTRUMENT assert that the stamp was readable at capture, because `Unproven != Unproven` is false and would have passed the leg by having nothing to compare. Mutation-proved both ways: an original that survives the ceremony REDS the identity assert, an unreadable stamp REDS the instrument assert. **No flake-registry.json entry** — its `retire_when` is "after the owning defect is fixed", the fix ships in the same commit, so an entry would be born retired; and registry entries require a same-SHA-rerun confirmation, which this never had (doyle's RCA proved it from the log, not from a rerun) |
| `spt::psyche_real_bound_kill_soft_budget_e2e` | assert-FAIL 1× hfenduleam Windows golden (2026-08-04 run 30940180764 att1, sha 17f95a7, :309 — expected `Some(Ok)`, got `Timeout` 1s×11 on summarizer.bat); LEAK 7/7 across recent golden Windows legs (7th on the att3 GREEN leg, 17.497s) | **two defects, not one signal** (hertz 2026-08-04): (1) assert arm — the positive control (:309) fires the FAST body through the SAME 1s kill bound; `wait_bounded` starts its deadline AFTER spawn (runtime.rs:941) so 1s covers the child's whole startup, and a trivial `@echo` bat can exceed it under load (mechanism, not yet a measurement — the control leg's wall time is the instrument); (2) leak arm is STRUCTURAL and distinct — `wait_bounded` kills the DIRECT child only (child.kill() :949 on cmd.exe); the sleeper is a GRANDCHILD (`cmd /C summarizer.bat` → ping), so ten kills leave ten ~5s survivors against nextest's 100ms leak timeout; no bound-shortening can fix it. USHER-chain delta-eliminated by dependency direction (spt-daemon below spt, deployah source-verified). Kin: releases#90 echo-commune spawn-killed-at-bound | DISPATCHED hertz 2026-08-04 (pkg item 1): control-leg bound via `BrainLifecycle::refresh_manifest` (lifecycle.rs:681) SAME host — budgets are host state (`Arc<Mutex<PsycheBudgets>>` :672) so the standing latch and the `stamp()==None` clear row stay load-bearing; a second host would make that row vacuous. Leak fix lane-split doyle-ruled: TEST-ONLY sleeper fixture bin in spt-daemon = hertz now; PROD `wait_bounded` tree-kill-on-timeout = EVAL request to board, not dispatched |
| `spt::engine_room_bringup_e2e a_cleanly_offline_engine_room_comes_back_when_knocked_awake` | 1× kitsubito Linux golden (2026-08-04 run 30940180764 att1, sha 17f95a7, :1198 — erhost not up in 30s; brain stderr EMPTY vs named `ENGINE_ROOM_SPAWN_FAIL` emitter) | test file brand-new on the USHER branch. The chain's own suspect commit d449ce5 (exit-in-thread) source-eliminated on two strong legs: TEMPORAL (rc blocked in the bring_up RPC the whole window, rig-killed after — teardown never reached) + ARTIFACT OWNERSHIP (the awaited artifact is the broker's in-memory `sessions.insert`, broker.rs:6138-6153 — daemon-side chain, no CLI participant). Offline arm differs from the passing control arm ONLY by prior warm bring-up + `endpoint stop` | observed 1×; did NOT repeat at att3 same-sha (heavy-leg population 177 matched, the test RAN and passed — non-vacuous green). Declared read: next occurrence gets the stop-survivor broker-state RCA (wake_inflight gate broker.rs:5782-5852, `WAKE_DEDUP_WAIT=2s`; sessions reap; ceremony/perch re-read :5600-5604), not a rerun |
| `spt::resident_service_e2e` teardown LEAK at :664 (second svcmock outlived the sweep) | 1× hfenduleam Windows golden (2026-08-04 run 30940180764 att2, sha 17f95a7): second `svcmock.exe` pid 14452 outlived teardown; functional half all-TRUE, 3 reap verdicts Killed | teardown/zombie class (IR-34 kin). DISTINCT from this test's :382 PRECONDITION signature (ir15 att1, daemon never came up in 124s — that one is IR-17's family and carries no defect evidence): 2-of-11 recent Windows victim legs by test, 1-of-11 by this assert | DISPATCHED hertz 2026-08-04 (pkg item 4). Did not recur at att3 (passed, no leak row). +1 UNCLASSIFIED sighting 2026-08-17: `a_declared_service_rises…` FAILED 25.7s in a 745-full-parallel local sweep (nameplate-asm @2564f93, agent env scrubbed) — failure body NOT captured (`tail` ate it; labelled hole, no signature to compare), targeted rerun ×3 green. Counted as a sighting of SOME row of this test, attributable to none |
| `spt::engine_room_bringup_e2e a_cleanly_offline_engine_room_is_still_brought_up_by_its_own_gate` — :1212 MINTED-NOT-RESUMED (sid identity) | 1× hfenduleam local assembly gate (2026-08-17, worktree nameplate-asm @2564f93, QUIET targeted round 1-of-3, 6.85s; rounds 2-3 green; +1 earlier FAIL 53.9s in the same tree's 745-full-parallel sweep, body tail-eaten — that occurrence carries no signature). Hot reproducer: **4–5/20** in quiet single-cell isolation at `f2d215a`; baseline arm at `60d74ea`: **5/20**. The cross-sha rate proves the family pre-existed the milestone head. | **TEST DEFECT — assert-side sampling race, not stopped-seat re-adoption.** Failure captures held `sid_after == sid_before` while broker truth crossed `None → Some(2)`, warm bring-up spawned, and stop returned `STOPPED`. The engine-room path unconditionally mints a fresh label (`broker.rs:5643`) and launches it with `is_resume=false`; the broker inserts its numeric PTY session into `sessions` before replying `Spawned`, while the child stamps the minted label only on its later self-bind. The test waited only for broker-table visibility and immediately sampled `info.json`, so it intermittently read the stopped life’s deliberately retained sid before the fresh bind rotated it. The alternate wake/reconcile path is ineligible: `endpoint stop` terminal-normalizes to `offline` + `suspended`, while `resume_woken_endpoint` requires active rest intent. Probe runs ended less than one second after admission and showed zero seat-gate involvement, matching the boundary. NAMEPLATE remains source-eliminated: no broker/session-mint/erhost file in that delta. | **CLOSED — GATE PASS** on `test/keystone-182-family-b` @`541e56f` (code @`a58a559`): after broker truth appears, poll under a hard 10s bound for the observable sid rotation before killing the controller. A fresh self-bind passes; unreadable, never-bound, or genuinely re-adopted records return the still-equal observed sid and remain red. Focused cell 1/1 green (6.97s), focused Clippy `-D warnings` green, traceability 776/776. The pre-registered signature-classified gate on the lane tip ran the exact cell ×20: **family-B sid signature 0/20** versus the hot 4–5/20 baseline, proving the fix effective. All four failures were the independent pre-#197 family-A throttle signature at 61–63s — expected on this `f2d215a`-based lane and not counted against this fix. Rides the next assembly as the test lane. |
| `spt::engine_room_bringup_e2e a_cleanly_offline_engine_room_is_still_brought_up_by_its_own_gate` — :1150 PRECONDITION (warm bring-up never spawns) | 3×/3 hfenduleam main-checkout pool (2026-08-17, main @27d40b9, agent env scrubbed, 186s each — consistent, not intermittent) | **TEST INFRASTRUCTURE DEFECT, not product evidence (IR-21/IR-39 class).** The clean main target had no `target/debug/mock-session.exe`: `cargo test -p spt` builds the `spt` test target but not another package's `mock-adapter` binary. `engine_room_bringup_e2e` derived that absent sibling path without asserting it, registered it as the harness command, then spent the full launch bound waiting for a program that never existed. This fully explains `warm bring-up spawned=false`, `endpoint stop` → `NO_SUCH_ENDPOINT`, the empty brain stderr, the 186s duration, and why a warm assembly pool passed or reached a later assertion. Explicitly building `cargo build -p mock-adapter --bin mock-session` restored the real loop; the earlier environment-sensitivity hypothesis is retired. Distinct from the genuine offline/control-projection race and from row 41's Linux occurrence. | CLOSED by test-only hardening: fail immediately with the missing path plus actionable build command, and prebuild the cross-package fixture before the golden suite. No product policy changed. RCA independently source-confirmed by doyle 2026-08-17 |
| `spt-daemon daemon::tests::a_tree_teardown_reaches_a_grandchild_the_service_spawned` | 2× hfenduleam Windows golden: 10.176s run 30776330383 @184f2ac (2026-08-03); 10.225s run 30873007187 att1 @4b37512 (2026-08-04) | **TWO mechanisms wearing one assert string — do not count as a cross-sha repeat** (doyle source verification 2026-08-04): red 1 was the bare-pid `provably_gone(grandchild)` poller reading a pid-reuse stranger — it is the cited motivating incident in the pin comment (daemon.rs:2726-2737) and was RETIRED by a5042ec ("poll the process we pinned, not the number it happens to hold"); red 2 is POST-fix (pin present at 4b37512:2371) — a pinned, stamp-checked identity stayed not-provably-gone for the FULL 10s bound (daemon.rs:2757), mechanism OPEN (H1 unauthenticated SELECTION / H2 enrollment gap — see INFRA-REGISTER IR-17 sub-observation). Both reds sit AT the bound; todlando's off-CI discriminator ran 0/200 under live-fleet load with every pass 1-4s (never near the bound; filter positive-controlled via `nextest list` = exactly one test). The lesson this row carries: assert-body diffing across shas carried the string across a mechanism change — identity must include the polled predicate's SUBJECT, not just the message | INSTRUMENTED-AWAITING-FIRE (hertz selection probe, `SELECTION_PROBE` line rides main; a green retires nothing — doyle-ruled). Green at att3 17f95a7. The bound question (is 10s right for a loaded runner) parked with hertz pkg item 4; kill path and spawn flags untouched until the probe fires (#131 stays todlando's, operator-triage-gated). **GREEN-CAPTURE BASELINE (hertz item 3, 2026-08-04, doyle-scoped: run-local flags, N=20, this row only).** Repro: `cargo nextest run -p spt-daemon --lib --success-output immediate --test-threads 1 -E 'test(a_tree_teardown_reaches_a_grandchild_the_service_spawned)'`, ×20, on a released box (no CI worker, no release build), agent env stripped, lane `test/probe-green-capture` @`60ca8f8` base `6ec5237` — source-identical to the v0.54.0 tag `86f0d84` for this row (the range is the release commit only: Cargo.toml/Cargo.lock/CHANGELOG.md, **zero `.rs`**). **field 4 (ppid-match count) = 2 on 20 of 20 runs. min 2 / median 2 (lower-middle, convention pre-registered BEFORE the capture) / max 2. Raw: 2×20. ZERO variance.** All 20 exits 0; durations 1.86–2.79s (consistent with todlando's 1–4s passes, nowhere near the 10s bound). Population asserted per run (`Starting 1 test` ×20) and the 20 probe lines carry 20 UNIQUE candidate pids, so these are 20 distinct runs, not one line re-read. Both sweep populations 0 after: pool-scoped and the System32 `cmd`/`ping` victims. **What this establishes:** a second process in the child's parentage is STRUCTURAL here, not an excursion — hertz's prediction (pre-registered with doyle before the capture, falsifier `min=median=1`) SURVIVED. Mechanism named in the same lane's comment fix: this row spawns WINDOWLESS, which masks `DETACHED_PROCESS` back off (daemon.rs:1218-1219) so the child owns a console. **What it does NOT establish, and must not be read as:** (a) the companion's IDENTITY is UNMEASURED — the probe renders only the CHOSEN candidate and the reject census prints only on the panic path, so "the second match is a conhost" remains inference, not data; a red is what would print it. (b) The floor here is **2**, so the incident's `3` is an excursion above 2, **not** above the `1` the comment and register carry — on this box, at this sha, "it had always read 1" is not what the scan reads. That figure needs re-deriving on the hfenduleam runner before anything is built on it; a 20/20 zero-variance local read cannot speak for a different box. (c) A green still retires NOTHING (doyle's standing rule) — this is a baseline SHAPE, not a fix, and the row stays INSTRUMENTED-AWAITING-FIRE |