--- name: update-wedge-2-resume-steal description: UPDATE-WEDGE saga β€” v0.30.5 ALSO field-failed; hertz RCA PINNED real root (2026-07-09) = controller writer CTRL_WRITE LOCKED-never-DONE on a black-holed rc --take consumer, load-gated. doyle gates the eventual fix. (History below = 4 falsified doyle theories incl round-3 SharedSend deadlock.) metadata: node_type: memory type: project originSessionId: 6402bb1f-a8b9-46ab-804f-1ebe6272da0a --- **🎯 ROOT PINNED BY hertz (2026-07-09) β€” v0.30.5 field-failed (4th failure), RCA handed to dedicated agent hertz; doyle now supplies field evidence ONLY, hertz owns root, doyle gates the fix. DO NOT propose a 5th theory. DO NOT re-anchor on the 4 falsified roots below (steal / IPC-starve / reap-drive / SharedSend-deadlock).** Mechanism (hertz, code+capture-proven, load-gated): controller writer hits `CTRL_WRITE_WAIT β†’ CTRL_WRITE_LOCKED` but **never `CTRL_WRITE_DONE`** because the `rc --take` consumer on the far end is black-holed/unresponsive β†’ the write blocks >15000ms β†’ `BRAIN_SUBSCRIBER_STALL_EVICT`. seq0 boot phase is CLEAN (write_us 10-14us result=ok); wedge fires only when epoch=2 controllers stream real multi-KB frames (seq:19-174) into a dead consumer. Decisive repro capture `C:\Users\decid\AppData\Local\spt-core\rca-active-20260709-222659` (deliberately black-holed rc --take consumers): fresh cold restart pid 31192 active.log L6893 clean seq0 conn3-6 (sid1-4 = deployah/doyle/flynn/lia), then epoch=2 four-way stall-evict burst L8234-8237 (sid 1/4/3/2). `status.active.json` = INTENTIONAL pre-recovery snapshot (broker_image 0.30.0, broker_stale=true, stall_evict_count=8) β€” kept as artifact; hertz's healthy 0.30.5/false/0 was a separate post-recovery live check on signed prod binary. Standalone bug found in capture (queued, delivery-integrity cluster, NOT the wedge root): `echo_commune.rs:71` unsafe byte-slice panics mid-UTF8-char on multi-byte content (em-dash). **STATUS 2026-07-09 β€” hertz handoff RECEIVED, doyle ruled GO, todlando DISPATCHED (round-4 build in flight).** hertz root doc = `docs/UPDATE-WEDGE-ROOT-CAUSE.md` on branch `triage/update-wedge-rca-capture` @ `f8596ca` (anchors @5d3723f). SHARPER root than my memory block above: it's a blocking OS pipe write, NOT the mutex convoy β€” `controller_writer` holds `SharedSend=Arc>` guard ACROSS `write_frame`; Windows `interprocess 2.4.2` β†’ `WriteFileEx`+`SleepEx(INFINITE)`, NO write timeout (`set_timeout`=Unsupported); non-draining `rc --take` consumer blocks it forever; logical `stall_evict_controller` removes the ROLE but never `CancelIoEx`s the write or closes the conn β†’ writer wedged ~128s. `CTRL_WRITE_LOCKED wait_us=0` EXPLICITLY rejects the convoy (block is inside the OS write, not the lock). Why v0.30.5 missed it: cursor-only killed the daemon-conn subscription convoy but a REMOTE take-controller's own pipe write is a separate unbounded path. **FIX (accepted): broker-side bounded+cancelable+poison-on-fail framed-write primitive routing ALL physical-conn writes (controller/viewer/dispatch-reply/terminal-control)** β€” Arc-owned conn obj w/ write gate + poison state + per-op identity + CancelIoEx; broker-side only, broker restart to load, no wire change, Unix preserved. Activates round-3-deferred REQ-HAZARD-SHAREDSEND-NO-BLOCKING-WRITE-UNDER-LOCK. **doyle gate conditions (docs/UPDATE-WEDGE-RCA-DISPATCH-todlando.md):** RED-first Windows test `brain_decouple.rs::non_draining_controller_stall_evict_releases_writer_and_connection` (reuse suspended_brain_controller... helpers @:222, 9-step seq in root doc); MANDATORY real Windows integration leg (CancelIoEx fix β€” Linux pass meaningless); clippy/traceable/nextest 0; Unix no-regress. **doyle field-acceptance leg:** `scripts/rca/blackhole-controller.ps1` (in f8596ca) against disposable endpoints, operator-supervised, REDβ†’GREEN. **r4 BUILD PROGRESS (todlando, branch `fix/update-wedge-bounded-conn-write`, worktree .worktrees/uwedge-fix):** conn.rs `BrokerConn` landed, ALL write sites routed (grep sweep = zero raw send.lock()+write_frame, SendHalf never escapes conn.rs), independent Condvar watchdog β†’ CancelIoEx/shutdown at the bound. MONEY TEST GREEN: `non_draining_controller_stall_evict_releases_writer_and_connection` PASS (RED-first confirmed pre-fix). W2 regression found+fixed (deadline-retirement recorded in the SAME BRAIN_SUBSCRIBER_STALL_EVICT tally, TimedOut-only). Physical bound wired to the SAME `brain_write_deadline()` knob (one knob, no new env). **doyle RULING on false_promote.rs (docs/UPDATE-WEDGE-R4-FALSE-PROMOTE-RULING.md): Option C β€” REJECTED todlando's option A (test-only knob splitting logicalβ‰ physical = prod-divergent config, gate-integrity smell). C = re-source the rig's wedge from a real-parked-writer (which the watchdog now retires) to a STATICALLY-held write_blocked_since (seed_controller_with_block fixture broker.rs:5496 β€” no BrokerConn::write, no armed watchdog β†’ wedge persists); keeps rig's exact shape, tests the GATE DECISION in isolation while brain_decouple tests the watchdog. Feasibility caveat: fixture is #[cfg(test)]-private to broker.rs, needs a small test-visible injection seam. Fallback = B via OBSERVABLES (promote-after-retirement ordering + assert-promote-lands), NEVER a tight-sleep window. Concurred with todlando's "22:47 class dead by construction" (watchdog guarantees drain within bound; DRAINED gate retained as 2nd line). **βœ… CODE GATE PASS (doyle, 2026-07-10, SHA f10e5e9, branch fix/update-wedge-bounded-conn-write pushed origin off main@5555648).** Isolated worktree + fresh CARGO_TARGET_DIR: clippy --workspace -D warnings=0, traceable=0, nextest -p spt-daemon 609/609 passed 0 failed (5 leaky = pre-existing brain-trial sleep-children, NOT r4). MONEY TEST `non_draining_controller_stall_evict_releases_writer_and_connection` (#[cfg(windows)], hertz 9-step) PASS; cross-platform companion PASS; false_promote Option C (test_seed_wedged_local_controller static seed) PASS; brain_swap PASS. todlando Linux gravity 46-target sweep GREEN (1 pre-existing fixed-port AddrInUse, alone-pass). Option C verified in-diff. REQ mint + KNOWN-HAZARDS 7.38 + checklist row + tags all present. Verdict doc docs/UPDATE-WEDGE-RCA-DISPATCH-todlando.md ##GATE VERDICT. **REMAINING = field-acceptance (blackhole-controller.ps1) β€” needs the FIXED binary as the box daemon (box=v0.30.5 unfixed for r4) β†’ daemon bounce onto f10e5e9 + disposable endpoints = operator-coordinated. Release is outward-facing β†’ doyle HOLDS for operator release-sequencing ruling (A: field-verify-before-publish / B: publish-then-field-verify per the saga's established broker-side pattern). Does NOT GO deployah unilaterally. deployah on deck.** doyle NEXT: operator rules sequencing β†’ field-acceptance green β†’ GO deployah cuts v0.30.6.** **⚠️ RELEASE-BASE CATCH (doyle, 2026-07-10) β€” caught at GO, before publish:** operator ruled publish-then-field-verify + GO'd deployah. BUT fix branch f10e5e9 is based on STALE main `5555648` (pre-round-3) β†’ MISSING v0.30.3/4/5 (confirmed: resume_session_cursors count 0 vs 2 in origin/main@0a306c9; round-3 REQ absent). A DIRECT cut from f10e5e9 would REGRESS round-1/2/3 (reintroduce the SharedSend brain-conn deadlock). Trial `git merge-tree 0a306c9 f10e5e9`: CLEAN except ONE conflict β€” traceable-reqs.toml tail (both append REQ blocks β†’ resolve = keep BOTH). All code merges clean (broker/conn/nethost/brainproc/brain; conn.rs is new). **CORRECT RELEASE PATH (sent deployah): base release branch OFF origin/main@0a306c9 + MERGE f10e5e9 in (keep both REQ blocks) β†’ v0.30.6 counter 57 β†’ the MERGED-tree release-PR CI (Windows leg / money test on hfenduleam) is the gate on the COMBINATION (my local gate certified r4-in-isolation only) β†’ publish only on merged-CI-green β†’ operator applies β†’ doyle field-verify via blackhole-controller.ps1.** LESSON: branch fixes off CURRENT origin/main, not a triage-branch tip; ALWAYS check release-base ancestry (merge-base --is-ancestor published-tag fix-sha) before GO β€” a green gate on a stale base ships a regression. **+ deployah caught a DUPLICATE REQ id my trial-merge missed (2026-07-10):** `REQ-HAZARD-SHAREDSEND-NO-BLOCKING-WRITE-UNDER-LOCK` exists in BOTH main/0a306c9 (round-3's DEFERRED Option-C stub, required_stages=[]) AND f10e5e9 (r4 ACTIVATED [doc,impl,int]). git text-merge won't flag it (different regions) but traceable would (dup id). RESOLUTION (todlando doing the merge): keep r4's ACTIVATED block, DELETE main's deferred stub, keep round-3's distinct REQ-BRAIN-RESUME-NO-CONN-DEADLOCK. My earlier "keep both REQ blocks" was WRONG for that id. LESSON: on a merge, grep for DUPLICATE ids across both sides, not just git conflicts. **+ DEPLOYMENT STORY (confirmed hertz root doc Rollout + KNOWN-HAZARDS 7.9): v0.30.6 is BROKER-SIDE (conn.rs) β†’ RESTART-REQUIRED to load (a seamless brain-only `fetch --apply` keeps the OLD broker, does NOT activate a broker-side fix; ADR-0004 forbids auto-kill). Apply via `apply --finish` or `daemon stop`+start. CONTRASTS round-3 v0.30.5 (brain-side=seamless). CHANGELOG must say restart-required; the field-verify daemon-bounce IS the loading restart.** FLOW: todlando merges f10e5e9 into origin/main@0a306c9 + resolves traceable + re-gates β†’ merged SHA β†’ doyle gates the MERGED tree (ships-what-ships, not f10e5e9-isolation) β†’ deployah cuts v0.30.6 counter 57 β†’ publish β†’ operator applies via full restart β†’ doyle field-verify (blackhole-controller.ps1, disposable endpoints). **βœ… AUTHORITATIVE GATE PASS on MERGED tree 53cfab5 (doyle, 2026-07-10) β€” deployah GO'd for v0.30.6.** clippy --workspace -D warnings=0; traceable check=0 (472/472, SHAREDSEND deduped to one activated block); nextest -p spt-daemon merged 622/623. Sole full-suite fail = the MONEY TEST non_draining_controller_stall_evict (12.18s) under a **100%-FULL DISK** + 622-test contention β€” EXONERATED: isolated rerun **3/3 PASS 0.75s, stall_evicts_seen=1**; matches todlando's merged PASS (1.53s) + my f10e5e9 gate (2.19s). CLASSIFICATION KEY: the r4 CORE fields were GREEN even in the FAILING run (CONN_WRITE_POISONED fired, handler_released=true, old_client_write_is_err=true, resume=Controller) β€” only the `stall_evicts_seen` KIND_SESSIONS-reap-observation momentarily read 0 (reap sampling lagged past the assert window under load; physical bound retires at 400ms fast). false_promote Option-C + brain_swap + companion + inject_control_wedge collision all PASS on my run. **TWO box-health/test findings flagged: (1) HFENDULEAM disk hit 100% (0 free) mid-gate β€” os-error-112 killed the first nextest build (NOT code; kin [[hfenduleam-disk-full-ci]]); freed 17G+14G of my gate targets; told deployah a release-CI os-112 = disk-full-not-code, don't re-tag. (2) TEST-ROBUSTNESS FOLLOW-UP (todlando, post-release, not a blocker): the money test's `stall_evicts_seen==1` assertion flakes under CPU/disk load β€” needs a bounded-retry/wider poll window; queue with the pre-existing fixed-port/live-daemon test-isolation cluster (inject_control_wedge etc.).** NEXT: deployah cuts v0.30.6 counter 57 off 53cfab5 (restart-required changelog) β†’ CI green on merged tree β†’ publish β†’ operator applies via FULL daemon restart β†’ doyle field-verify (blackhole-controller.ps1, disposable endpoints, expect CONN_WRITE_POISONED + stall-evict within bound + fresh rc --take recovers no-restart). **⚠️ RELEASE RE-BLOCKED (2026-07-10) β€” deployah CI caught a REAL merged-tree fail my local gate MISSED. v0.30.6 PR #75 HELD, NOT merged.** kitsubito Linux CI leg: `brain_resume_conn_deadlock::nondraining_subscriber_starves_under_flood_regression_guard` @:280 FAIL ("worst completed 10 ... if subscriber write became non-blocking (Option C) this guard is stale"). This is round-3's NEGATIVE-SPACE proof that a non-draining subscriber deadlocks (worst ≀5 ticks best-of-3) β€” r4's Option C BOUNDS every physical-conn write β†’ the blocked subscriber write is retired at the deadline instead of holding SharedSend forever β†’ heartbeat no longer starves β†’ assertion invalid. **The test author PREDICTED this staleness in the panic text.** MY GATE MISS: it passed SPURIOUSLY on my single local merged run (best-of-3 timing luck under low contention); kitsubito's timing exposed it β†’ lesson: timing-based best-of-K guards pass spuriously per-box, CI's different env is a real second gate. **doyle RULING (docs/UPDATE-WEDGE-R4-STALE-GUARD-RULING.md): guard STALE BY DESIGN β†’ RETIRE it, do NOT rerun, do NOT revert cursor-only resume (still correct β€” post-r4 subscribing would POISON the brain conn). REPLACE with the round-3 FAST-FOLLOW `daemon_resume_leaves_zero_brain_subscribers` (in-proc deterministic: resume_session_cursors() β†’ assert Arc 0 viewers/session; timing-INDEPENDENT). Keep the 2 positive cursor-only heartbeat tests. Tests-only, no product change, r4+round-3 untouched, closes the round-3 wire-caveat.** FLOW: todlando swaps the guard off 53cfab5 β†’ re-gate β†’ new SHA β†’ doyle re-gates (watching this test) β†’ deployah re-cuts v0.30.6 off new SHA. deployah holding PR #75. Windows disk-112 moot until this lands. **βœ… GUARD-SWAP GATE PASS @3a4335c (doyle, 2026-07-10) β€” deployah GO'd to RE-CUT v0.30.6 off it (close PR #75, fresh cut).** 3a4335c = 53cfab5 + tests-only (retire stale starvation guard β†’ deterministic `daemon_resume_leaves_zero_brain_subscribers` + a paired β‰₯1 sensitivity arm so it can't go vacuous; retire dead ResumeMode::Subscribe arm) + one additive #[doc(hidden)] `Broker::test_session_viewer_count` read seam. PRODUCT CODE IDENTICAL to gated 53cfab5. GATE: clippy -D warnings=0, traceable=0, nextest 623/623 passed 0 FAILED; new guard 5/5 determinism reruns (0.4-0.85s, rock-stable β€” no flood/timing); money test PASS in-suite + 3/3 isolated. **KEY: the money test passing 3/3 now (disk healthy) RETROACTIVELY CONFIRMS the 53cfab5 money fail was PURE disk-contention flake** β€” root was ~81GB stale gate-worktree target/ bloat (todlando reclaimed 57G; my scratch targets the rest). Both r4 int gates clean. LESSON REINFORCED: (1) timing-based best-of-K guards pass spuriously per-box β†’ CI's different env is a real 2nd gate (caught the stale starvation guard my local run missed); (2) 100%-disk fakes test fails (os-112 build-kill AND contention-timeouts) β†’ free disk + rerun, kin [[hfenduleam-disk-full-ci]]. NEXT: deployah re-cuts v0.30.6 counter 57 fresh off 3a4335c β†’ CI green β†’ publish β†’ operator applies FULL restart β†’ doyle field-verify (blackhole-controller.ps1). Symptoms handoff docs/UPDATE-WEDGE-SYMPTOMS-FOR-RCA.md; falsified-theory history docs/UPDATE-WEDGE-2-ROUND3-*. [[ground-dont-assume-on-incidents]] [[dont-solo-across-role-lines]] **⏳ v0.30.6 RELEASE-CI SAGA (2026-07-10, PR #76 off 3a4335c) β€” kitsubito INCIDENT-BOX root pinned + REMEDIATED by doyle.** Linux CI leg red twice (Windows leg + n1-gateΓ—2 + traceability all green incl money-test). Fails = timing/e2e-load class: net_worker_starve canary 342ms/300 (doyle ruled load-flake β€” cleared <300 in my idle 623/623 gate, plain #[test] so it RAN), pumpdeadline 2ms-early wake (timer slack, passed on rerun), + 4 `spt::` e2e "PRECONDITION: brain never came up" bringup-timeouts (bind_honest_cross_perch / live_adapt_translation_swap Γ—2 / multi_subnet_bringup). **DECISIVE pre-r4 proof: main CI at the v0.30.5 merge push (run 29063755424) already failed 7 of the SAME e2e class on kitsubito, 2 exact same names β†’ class pre-dates r4, r4 adds zero red.** ROOT = kitsubito resident daemon: ancient binary reporting **spt 0.6.0** (the known kitsubito 0.6.0 debt, Jun-13 mtime) busy-spinning **146% CPU + 26% MEM for 3d9h** (PAIR_MEET_UP:SPT_DEV rendezvous loop @step 59.4M; brain cycling PEER_PUMP_FAIL "brain IPC read deadline elapsed"β†’restart) + a transient load-24 spike. **REMEDIATION (operator-authorized): doyle upgraded it to v0.30.5 musl** (systemd --user `spt-daemon.service`; SPT_HOME=~/.spt-core; sha-verified from spt-releases; old binary kept ~/.local/bin/spt.old-12) β†’ CPU settled ~3%. 0.6.0 debt RETIRED. CAVEAT: swap landed MID deployah-rerun β†’ that run tainted; if red, ONE more rerun on the clean box is the decider. LESSONS: (1) check the RUNNER BOX (top CPU + resident daemon version) before burning CI reruns on timing fails; (2) compare vs main's CI on the previous merge β€” pre-existing red class = decisive not-my-diff proof; (3) messages crossing on the wire β†’ re-sync state explicitly, deployah+doyle each retracted one stale directive. **THEN (same saga): clean-box decider rerun surfaced a REAL deterministic blocker the starvation had been masking β€” clippy `needless_return` @ conn.rs:215 (sever()'s idempotence-guard return; the #[cfg(windows)] DisconnectNamedPipe block after it strips on Linux β†’ return becomes fn-final). NOT toolchain drift (kitsubito rustup mtime Jun-7, untouched β€” deployah's theory falsified by mtime check); why-now = earlier CI Linux clippy passes were INCREMENTAL-CACHE lint suppression + doyle's gate clippy-0 was WINDOWS clippy (cfg block present = lint invisible). GATE LESSON (binding for doyle): cfg-divergent code needs a FRESH-TARGET clippy -D warnings leg on BOTH platforms β€” Windows clippy proves nothing about Linux-stripped shapes. Ruled fix = allow(clippy::needless_return)-with-why-note (minimal churn on gated tree), todlando dispatched off 3a4335c.** Operator also authorized STOP+disable of kitsubito's spt-daemon entirely (box doesn't need spt-core now) β€” CI box fully daemon-free. **βœ… RE-GATE PASS @e0bd8b9 (doyle 2026-07-10): todlando's fix = 5 lines conn.rs-only, `#[cfg_attr(unix, allow(clippy::needless_return))]` + why-comment (BETTER than doyle's blanket-allow β€” keeps Windows lint coverage; restructure rejected as lint-roulette β†’ clippy::needless_if on the empty Unix if-body). Gate legs: Win fresh clippy 0 + traceable 0; LINUX clippy 0 on kitsubito fresh bundle-clone+target (exact CI step β€” private repo: https clone needs auth, use git bundle over scp, bundle needs a REF not bare sha); money test PASS 0.788s + companion. deployah GO'd to re-cut off e0bd8b9.** NEXT: deployah closes PR #76, re-cuts v0.30.6 counter 57 β†’ CI green ALL legs β†’ merge/tag/publish β†’ operator FULL-restart apply (broker-side fix) β†’ doyle blackhole-controller.ps1 field-acceptance (expect CONN_WRITE_POISONED + BRAIN_SUBSCRIBER_STALL_EVICT ≀15s, conns terminated, fresh rc --take recovers, NO daemon restart) β†’ round-4 CLOSES. --- **Incident (2026-07-09, HFENDULEAM, post-v0.30.3):** operator ran `spt update fetch --apply` (0.30.2β†’0.30.3) with 7 spt-hosted broker PTY sessions live. The v0.30.3 counter-54 fix WORKED (brain PROMOTED cleanly, no 30s-timeoutβ†’rollback, BRAIN_RESUMED re-attached 7 + BRAIN_PROMOTED v54) β€” BUT all 7 sessions FROZE (output streaming stopped entirely, not just control lost) and `spt rc` couldn't retake any. Operator force-restarted (`daemon stop`+`start`) β†’ clean cold-start 0.30.3, all healthy. **Steady-state 0.30.3 is fine β€” the bug is strictly the brain-respawn-under-live-load path. No rollback needed.** Extends [[update-wedge-rca-v53]]. **ROOT (LOCKED, operator-confirmed + code-cited):** `resume_sessions` (brain.rs:985) re-attaches EVERY broker session `KIND_SESSIONS` returns as `subscribe_with(AttachIntent::Control, by:None)` (brain.rs:1450-1455). The `subscribe` docstring's "a None identity never displaces (falls back to viewer)" is a **MYTH** β€” true ONLY against a DIFFERENT-REMOTE incumbent. `resolve_subscribe` (broker.rs:1134-1153): stall-evict FIRST β†’ `become_controller` if slot free OR same-local-identity (None==None). `become_controller` (broker.rs:883) on the same-local path fires **NO `Displaced` notice** (gate-#7 self-kick guard, broker.rs:542/1144). So on a box with N spt-hosted broker PTYs (each with a LOCAL by:None controller β€” the operator's local `spt rc` + the PTY's spawn-time controller), a brain-respawn **SILENTLY STEALS all N controllers**: the broker stops writing each session's output to the operator's rc sink (now writes to the brain's undraining sink) but does NOT close the conn β†’ **no EOF, no `PumpEnd`, no 30s reconnect/detach splash** (splash fires ONLY on `BrokerGone`/`Severed`, rc.rs:1671). β‡’ every session FREEZES immediately. The daemon brain drains NONE of these (hosts no PTY sessions, brainproc.rs:184). **The 5-vs-2 stall-evict split is ACTIVE-vs-IDLE, NOT remote-vs-local** (my + todlando's first guess was WRONG β€” operator corrected): the 5 producing output made the stolen writer block >15s β†’ `stall_evict_controller` (the field's STALL_EVICT :1:3:5:7:4); the 2 idle β†’ writer parked β†’ no block β†’ no evict, but still silently stolen+frozen. **Failure B (rc-retake globally deadlined):** the stolen brain conn head-of-lines the shared brain↔broker conn on the N-session replay β†’ journaled `rc` attach (`bounded_block_on`, nethost.rs:1083, REQ-BROKER-ATTACH-JOURNAL-RESILIENT/#16) deadlines ("brain IPC read deadline") β€” a DOWNSTREAM symptom of the steal. **Why v0.30.3 shipped this:** the counter-54 A'-rig was a SINGLE black-holed session β€” it proved the promotion strand but never exercised N LIVE controllers under replay during a swap. The steal is a pre-existing latent bug, hidden until N broker PTYs were present at a respawn. **FIX (v0.30.4, doyle-ruled, brain-side, minimal):** `resume_sessions` re-attaches as **`AttachIntent::Viewer`, NOT Control** (Viewer already exists, brain.rs:924/963). A viewer never owns `driven_by`, never calls `become_controller`, is never stall-evicted β†’ never steals/orphans the operator's rc β†’ output + control preserved across the swap (active AND idle). Control reserved ONLY for sessions the daemon brain genuinely drives (empty today β†’ all Viewer; forward-correct). Also fix the lying docstring. **REQ-BRAIN-RESUME-NO-CONTROL-STEAL** = [doc,impl,int] (unit dropped β€” branchless intent-flip, same FORK-2 call as [[update-wedge-rca-v53]]'s REQ). NOT the deferred CLEAN-CLOSE seed (that = OLD brain outgoing black-hole; this = NEW brain resume-steal). Failure-B rc-retake NOT gated (flaky subprocess-saturation; downstream of the steal; covered by Viewer-relief + #16 fail-prompt + operator-supervised field-verify). **GATE (the coverage the A'-rig lacked):** `crates/spt/tests/resume_no_control_steal_e2e.rs` β€” N REAL broker PTY sessions each with a HELD local (by:None) controller producing output-ticks; real `spt daemon brain` respawn/promote; DETERMINISTIC discriminator = a held controller's output-tick counter **freezes in RED (Control re-attach steals its sink) / keeps ticking in GREEN (Viewer)** + promotion still succeeds. GREEN passed (ticks gained=[18,18,18], promoted=true). This asserts the IMMEDIATE output-freeze (the real signal), NOT the flaky 15s stall-evict. **βœ… ROOT PROVEN + FIX GATED (round-3, v0.30.5, 2026-07-09) β€” SUPERSEDES both wrong roots below.** After TWO falsified mechanisms (my "IPC read-loop starvation" AND todlando's "NetHost 2-worker starvation" β€” both killed by rigs; net runtime EXONERATED, canary 32-33ms vs 15000ms), todlando's code-read PROVED the real root: a self-reinforcing **DEADLOCK on the brain conn's single SharedSend mutex** (broker.rs:78). The brain conn is BOTH a subscriber sink (resume subscribes N sessions) AND the req/reply channel; subscriber writers hold send.lock() ACROSS the blocking write_frame, and the dispatch heartbeat reply needs the SAME lock β†’ an actively-streaming session backs the conn up β†’ the non-draining daemon heartbeat's reply starves β†’ brain drains nothing β†’ controllers cross BRAIN_WRITE_DEADLINE 15s β†’ STALL_EVICT. **DEAD-PEER-INDEPENDENT.** counter-54 (ce053dc) only widened it; the Viewer fix couldn't heal it (brain still a subscriber, viewers have no stall-evict valve). **FIX = A-SPLIT (RULING 3b), brain-side SEAMLESS:** run_brain calls a NEW cursor-only `resume_session_cursors()` (seeds cursors, NO subscribe) β†’ heartbeat conn is req/reply-only β†’ no deadlock; `resume_sessions()`+resume.rs kept UNCHANGED for draining callers (D4-2 coverage intact). REQ-BRAIN-RESUME-NO-CONN-DEADLOCK [doc,impl,int] + deferred REQ-HAZARD-SHAREDSEND-NO-BLOCKING-WRITE-UNDER-LOCK (Option C, durable broker-side class fix, needs coordinated restart). **GATE PASS @03c71093**: clippy0, traceable0, brain_resume_conn_deadlock all 3 GREEN (both severities + nondraining-subscriber RED-arm), resume.rs+brain seams green; 1 red WAIVED = inject_control_wedge (seed-#9 head-loss, NOT in diff, env flake, green in v0.30.4 CI). doyle GO'd deployah CUT v0.30.5 counter 56 (bump-in-PR on fix-tip 03c71093), HOLD publish for operator field-verify (B). **STATUS 2026-07-09: βœ… v0.30.5 PUBLISHED β€” counter 56 LIVE @0a306c9.** CI GREEN (run 29061252708 @03c71093: testΓ—2 + n1-gateΓ—2 + traceability all pass, twohost skipped) β†’ deployah merged PR #74 β†’ origin/main @0a306c9, tagged v0.30.5. Operator ruled PUBLISH (reversing the earlier HOLD-for-field-verify): can't field-verify without a release to update TO (fetch--apply pulls Latest), and the fix is brain-side so applying v0.30.5 IS the field-verify. deployah published: Latest=v0.30.5, isDraft=false, 10 assets (4 signed metadata), update-set version:56 product 0.30.5 stable, 3 platforms. **NEXT = operator runs `fetch --apply` v0.30.5 on the wedge-prone HFENDULEAM box = the round-3 field-verify.** WATCH: NO ~15s post-promote freeze, NO STALL_EVICT, sessions keep streaming, rc attaches clean. Caveat: Latest may serve STALE update-set ~1min post-flip ([[release-updateset-stale-window]]) β€” re-poll after ~60s if daemon reports old version. PASSβ†’deployah finalizes ledger, round-3 done. FAILβ†’round-4 escalate. A is brain-side β†’ the v0.30.5 apply itself heals the box. Rulings/verdict docs/UPDATE-WEDGE-2-ROUND3-RULING.md (2/3/3b + GATE VERDICT); code-read docs/UPDATE-WEDGE-2-ROUND3-CODEREAD.md. Wire-gate follow-up (daemon_resume_leaves_zero_brain_subscribers, tests-only) β†’ main post-v0.30.5. **seed #9 ACTIVATED** into this milestone (operator directive; the flynn "ghost" + all-session ruling truncation ARE seed #9 live repro; also caused the waived gate flake). LESSON: 3 wrong roots before the code-read β€” report-before-fix + rigs caught each. [[ground-dont-assume-on-incidents]]. **[⬇ HISTORICAL round-3 trail β€” the SUPERSEDED wrong roots, kept as lesson:]** **⚠️ STATUS: v0.30.4 PUBLISHED but FIELD-VERIFY FAILED β€” Viewer fix correct but INSUFFICIENT; round-3 (v0.30.5) pending.** Operator-supervised `fetch --apply` 0.30.3β†’0.30.4 on HFENDULEAM (4 spt-hosted PTYs) **WEDGED AGAIN** (all froze, attaches deadlined; cold-restart recovered β†’ clean 0.30.4). The Viewer fix shipped in the bytes and DID kill the resume-STEAL (BRAIN_RESUMED re-attached 4 as Viewer, no become_controller) β€” but it was **aimed at the wrong root for the WEDGE.** Real root = **brain IPC read-loop STARVATION by peer-pump reply-read / unresponsive-peer dials** (gen-1 log: peers ecb39e40 + cfd9fc55 each burn the 10s submit-dial bound β†’ "peer reply-read: no progress within budget (brain IPC read deadline elapsed)" + PEER_PUMP_FAIL + broker STALL_EVICT:3, controller writer blocked >15s). **DECISIVE:** operator saw doyle-ONLY wedge TWICE on 0.30.3 STEADY-STATE (no update), recovered by `rc --take` β†’ a plain starvation window stalling the active-streamer's controller drain, NOT the resume-steal. New in v0.30.3/counter-54 (suspect ce053dc brain-self-drives-reap + pump reply-read, cf. [[subnet-peer-pump-stall-rca]]/PUMP-TRUTH). Round-3 fix = isolate brain IPC read loop from peer-op starvation (brain-side preferred) β†’ heals respawn + steady-state. Operator ruled: leave v0.30.4 Latest, ship round-3 fast. RCA in docs/UPDATE-WEDGE-2-RCA.md (## ROUND-3). todlando dispatched report-before-fix (pin shared-read-loop seam + isolation shape; doyle rules then builds). Lesson: 2nd time a green single-condition gate shipped a field-broken wedge fix β€” round-2 rig had a fast local broker + NO real peer dials, never reproduced the starvation. See [[ground-dont-assume-on-incidents]]. See [[v0304-published]]. **⚠️ ROUND-3 RIG VERDICT (2026-07-09, todlando) β€” net-runtime starvation seam FALSIFIED; RE-OPEN at brain drain.** Built doyle's asked-for net-runtime task-count + reactor-liveness-canary probe (nethost.rs, throwaway/no-REQ) + hermetic rig `crates/spt-daemon/tests/net_worker_starve.rs` (uwedge2 @14cb147). Fired K=12 dead-peer `submit_dial`s (BOTH shapes: proof-hang black-hole AND unreachable id-only = the field "peer unresponsive" shape) straight onto the 2-worker net runtime. **Canary stayed FRESH at 32-33ms vs the 15000ms stall threshold (~450x headroom); semaphore 12/24 free (never limiter).** The QUIC awaits YIELD cleanly β€” 2 workers multiplex 12 dead dials with zero reactor impact. **The 2-worker-starvation hypothesis (shapes A AND C both target it) does NOT hold.** DO NOT mint REQ-BRAIN-DIAL-BOUND-NO-STARVE / REQ-NET-LOOPBACK-NO-PEER-STARVE. Boundary map: loopback rc I/O (open_stream/send_stream via `handle().block_on` on a broker DISPATCH thread) proven healthy under the burst; field stall-evict = "controller writer blocked >15s on a suspended/black-holed BRAIN conn" = the brain MAIN LOOP not DRAINING (brokerβ†’brain IPC), NOT net-runtime saturation. `submit-dial exceeded 10s bound` field lines = benign expected dead-peer timeout. **RE-OPEN target = brain main-loop drain**; PRIME suspect = counter-54 reap-drive (ce053dc, runs in main loop ~500ms β€” fits the never-before-v0.30.3 steady-state doyle-only wedge with NO respawn); 2nd = respawn startup sequencing (net-consumers+peer-dials interleaved with entering the drain loop). Next rig (awaiting doyle's ruling) = brain+broker+PTY (brain_decouple template + real pump round w/ dead peer) asserting the reap-drive/sequencing blocks draining an active-streaming controller >15s. Verdict doc: docs/UPDATE-WEDGE-2-ROUND3-RIG-VERDICT.md. Reported to doyle, awaiting shape ruling. Lesson reinforced [[ground-dont-assume-on-incidents]]: rig BEFORE fix falsified a plausible seam both SMEs leaned toward. **⚠️ ROUND-3 CODE-READ (2026-07-09, todlando @2e395f0) β€” reap-drive EXONERATED; real seam = SharedSend self-deadlock on the brain conn.** doyle RULING 2 asked: does a dead peer lengthen the counter-54 reap-drive tick >15s? FALSIFIED by code: net_status (dispatch_net_status broker.rs:3573) = cheap field reads (no bounded_block_on); sessions() reap holds each OutputLog lock only briefly; append is try_send under the log lock (broker.rs:827); controller_writer holds the SOCKET mutex NEVER the log lock across its blocking write (broker.rs:1409-1411/1451 = the W2 decouple); viewers try_send+drop (1086). No dead-peer path into the tick (heartbeat 500ms vs deadline 15s). **REAL SEAM (code-proven deadlock):** the brain conn's single SharedSend mutex (Arc> broker.rs:78) is held ACROSS the blocking write_frame by subscriber writers (viewer_writer 1333/1342, controller_writer 1451) AND is the SAME lock the dispatch reply path needs (send_frame 4221 β†’ KIND_SESSIONS_REPLY/NET_STATUS_REPLY). The brain conn is BOTH a subscriber sink (resume re-subscribes N sessions onto it) AND the brain's request/reply channel β†’ when it backs up (brain only reads during the 500ms heartbeat's net_status/sessions), a subscriber writer BLOCKS holding send.lock() β†’ dispatch can't send the heartbeat reply β†’ net_status/sessions never return β†’ heartbeat stuck β†’ brain never drains β†’ deadlock β†’ stall-evict. Counter-54 added a 2nd per-heartbeat reply round-trip (sessions()) through the contended mutex = WIDENED the window (regression-consistent), didn't create it. Viewer-fix (v0.30.4) removed the steal but kept the brain a subscriber (viewer_writer same lock, NO stall-evict valve for viewers) β†’ deadlock persists = why v0.30.4 field-verify still wedged. **Dead peer NOT causal** β€” trigger is active-session OUTPUT backing up the brain conn faster than the 500ms heartbeat drains. **FIX SHAPE (brain-side/seamless, doyle to rule): (A) PREFERRED β€” resume_sessions does NOT re-subscribe (daemon brain hosts NO PTY sessions brainproc.rs:184 β†’ no consumer today β†’ the sub floods its own req/reply conn); (B) drain the brain conn on a dedicated Split reader (brain.rs:230, the pump's carrier) if subs must persist; (C) broker-side non-blocking subscriber write = DEPLOYMENT RISK (coordinated restart, daemon.rs:368), flagged.** A/B touch ONLY the brain β†’ seamless. Reported to doyle; HOLDING for shape ruling (A vs B) + REQ mint before building confirmer rig (RED-first, brain_decouple, both severities, dead-peer-INDEPENDENT) + fix. doc: docs/UPDATE-WEDGE-2-ROUND3-CODEREAD.md. **⚠️ ROUND-3 BUILD IN PROGRESS (2026-07-09, todlando; doyle RULING 3 = GO Option A, uwedge2).** Β§3 guard PASSED (Explore-verified: NO brain consumer reads the resumed subs β€” run_brain reads its conn only via net_status/sessions drain-and-discard; digest/relay/net-consumers/shellwake/presence use SEPARATE cold_start conns). REQs minted @2e29fcb: REQ-BRAIN-RESUME-NO-CONN-DEADLOCK [doc,impl,int] + REQ-HAZARD-SHAREDSEND-NO-BLOCKING-WRITE-UNDER-LOCK (Option C durable class-fix, deferred []). Confirmer rig crates/spt-daemon/tests/brain_resume_conn_deadlock.rs RED-first PROVEN: brain resume-subscribing + 6 flood PTY sessions β†’ heartbeat completes 0 ticks/6s = HARD DEADLOCK (dead-peer-INDEPENDENT); non-subscribing heartbeat under same flood runs free (control PASSES). Applied literal Option A (dropped subscribe_with in resume_sessions brain.rs:~1054) β†’ all 3 rig tests GREEN. **β›” BLOCKED on doyle: literal Option A BREAKS crates/spt-daemon/tests/resume.rs (3 tests HANGβ†’FAIL >144s β€” cold_start_resumes_all_sessions_from_the_broker_cursor etc.) which DRAIN resumed output (read_session_untilβ†’BrokerEvent::Output), asserting D4-2 output-continuity + carrying [unit->REQ-HAZARD-BROKER-PROCESS-ISOLATION]+[unit->REQ-DAEMON-2]. resume_sessions' subscribe is a TESTED forward-capability (a DRAINING consumer); it only deadlocks for a NON-draining consumer (the daemon heartbeat conn).** Flagged doyle 2 shapes (QUEUED): A-literal (drop subscribe, retire resume.rs delivery tests, coverage returns w/ Option B) vs A-split (RECOMMENDED β€” keep subscribing resume_sessions for resume.rs+B, run_brain calls NEW cursor-only resume_session_cursors(): seed cursors + return ids for BRAIN_RESUMED log, no subscribe β†’ daemon conn cursor-only β†’ deadlock gone, coverage intact). Both brain-side/seamless. HOLDING for doyle's shape ruling before touching resume.rs. NEXT after ruling: finish fix + adapt rig + doc note (CONTEXT/LIFECYCLE) + clippy --workspace 0 + traceable 0 + nextest (rig + resume + resume_no_control_steal_e2e + trial_drain_drive_e2e + brain_swap/false_promote/brain_decouple) β†’ ping doyle gate SHA. docs: UPDATE-WEDGE-2-ROUND3-{RIG-VERDICT,CODEREAD}.md + RULING.md ##RULING 1/2/3. **βœ… ROUND-3 BUILD COMPLETE β€” handed to doyle's gate (2026-07-09, todlando; doyle RULING 3b = Option A-SPLIT).** GATE SHA @03c71093 (branch uwedge2-resume-viewer). FIX: NEW `Brain::resume_session_cursors()` (seed cursors + return ids, NO subscribe) β€” run_brain (brainproc.rs) calls THIS instead of resume_sessions(), so the daemon's request/reply conn is cursor-only β†’ no undrained subscription β†’ SharedSend self-deadlock structurally impossible, brain-side SEAMLESS. `resume_sessions()` UNCHANGED (still Viewer-subscribes) + a DEADLOCK-WARNING docstring, kept for DRAINING callers (resume.rs + forward Option-B) so D4-2 output-continuity + REQ-DAEMON-2 + REQ-HAZARD-BROKER-PROCESS-ISOLATION coverage stays intact (A-split rationale β€” A-literal deleting that coverage until deferred-B was rejected per KNOWN-HAZARDS discipline). Confirmer rig crates/spt-daemon/tests/brain_resume_conn_deadlock.rs: cursor-only keeps heartbeat LIVE under 6-session PTY flood (both severities, deterministic GREEN) + non-draining SUBSCRIBER self-starves (best-of-3 N=12 regression guard β€” deadlock severity is LOAD-DEPENDENT: std Mutex unfair, hard-0 under contention / slips through quiescent, so best-of-K not single-shot). GATE (local uwedge2): clippy --workspace 0 | traceable 0 | nextest 9/9 daemon-seam (rig x3 + resume x3 + brain_decouple/brain_swap/brain) + 2/2 spt-e2e (resume_no_control_steal_e2e + trial_drain_drive_e2e exercise cursor-only run_brain). resume.rs UNTOUCHED+green. RED-first CAVEAT flagged to doyle: mechanism fully proven (subscribing-starves vs cursor-only-lives, same flood), but the run_brain WIRING line (cursor_only vs subscribe) is NOT independently RED-gated (rig calls methods directly; e2e don't flood) β€” a subprocess-flood wiring rig would be MORE timing-flaky; recommended mechanism-RED + one-line code-review as the coverage. REQs @2e29fcb: REQ-BRAIN-RESUME-NO-CONN-DEADLOCK [doc,impl,int] + REQ-HAZARD-SHAREDSEND-NO-BLOCKING-WRITE-UNDER-LOCK (Option C durable broker-side class-fix, deferred []). NEXT: doyle gates (isolated worktree + fresh CARGO_TARGET_DIR) β†’ deployah cuts v0.30.5. **FAST-FOLLOW (doyle-ruled, tests-only, AFTER v0.30.5 MERGES β€” HOLD until it lands, don't collide w/ gate/release):** small PR to main `daemon_resume_leaves_zero_brain_subscribers` β€” in-proc rig: run resume_session_cursors() on a Brain, assert the in-proc Arc reports 0 VIEWERS on each session (deterministic viewer-count truth, no flood/timing). Closes the wire-caveat (guards a future revert of brainproc.rs:202 back to resume_sessions). Regression-PREVENTION not current-correctness (wire verified correct now), so tests-only, no release. docs: UPDATE-WEDGE-2-ROUND3-{RIG-VERDICT,CODEREAD}.md + RULING.md ##RULING 1/2/3/3b. LESSON: 3-round report-before-fix chain FALSIFIED 2 plausible seams (net-runtime starvation, reap-drive blocking) before code-proving the real one (SharedSend brain-conn deadlock) β€” rig-before-fix + honest falsification paid off ([[ground-dont-assume-on-incidents]]). **Round-2 gate detail (fix is retained, not reverted):** Prior gate detail: **gate SHA @fb499db**, branch `uwedge2-resume-viewer` off origin/main @3136702. RED-verify PASSED (Control gained=[0,0,0] all frozen, assertion 1 fires / Viewer gained=[18,18,18] + promoted=[9]). Both DOCSTRING-FIX.md corrections folded (rig survivor-model = active-vs-idle; production subscribe docstring = silent-freeze). Local gate: clippy --workspace 0 | nextest 5/5 (rig + trial_drain_drive_e2e + false_promote + brain_swap + brain_decouple) | traceable exit 0. Handed to doyle's gate (isolated worktree + fresh target, verify RED bites prod) β†’ on PASS deployah cuts v0.30.4. Field-verify MUST be a real multi-session `fetch --apply`, OPERATOR-SUPERVISED (no repeat of the premature "shipped"). RCA docs: docs/UPDATE-WEDGE-2-RCA.md + docs/UPDATE-WEDGE-2-SME-todlando.md; ruling .worktrees/uwedge2/{FIX-RULING,DOCSTRING-FIX}.md. See [[ground-dont-assume-on-incidents]].