366	### 7.6 Pump brain-IPC reads must be deadline-bounded (a blocked read wedges the whole pump)  `[REQ-HAZARD-PUMP-IPC-DEADLINE]`
367	<!-- [doc->REQ-HAZARD-PUMP-IPC-DEADLINE] -->
368	- **Failure:** the peer pump is a SINGLE thread driving every leg (registry/notif/sync/update) against every peer over ONE brain-IPC client. Its reply reads (`net_open_stream`, `net_stream_send`, `net_dial`, and the sync/update pull `read_event` loops) were `loop { read_event() }` with no deadline. When a peer's QUIC path black-holes, the broker's stream-open/send awaits the dead peer and never sends the reply, so the brain's `read_frame` blocks FOREVER and the pump freezes mid-round. The heartbeat (loop-top) stops and the stall warning fires honestly, but `supervise_pump` cannot rescue it: the supervisor catches a panic / error / clean return, and a BLOCKED thread never returns. Observed twice — 2026-06-07, then a **2.2h wedge on hfenduleam (2026-06-11)** where the broker stayed responsive to `daemon status` while the pump thread sat dead.
369	- **Invariant:** in PUMP mode the brain carrier is **SPLIT at construction** — a dedicated `pump-ipc-reader` thread does blocking `read_frame` on the `RecvHalf` and forwards each framed result down a channel; the main thread writes on the `SendHalf` and reads with `Receiver::recv_timeout`. Every IPC reply read is bounded by a per-call **total-wait** deadline (`PUMP_PEER_IO_TIMEOUT` = 30s, > any legitimate round-trip, < the 60s QUIC idle; re-armed on stream progress for the streaming pull legs so a healthy long sync is never killed — only a ≥30s silence). **Mechanism note (Windows):** the carrier is a reader-thread + channel, NOT a non-blocking socket + poll — interprocess 2.4.2 on Windows named pipes has no portable read timeout (`set_recv_timeout` → `no_timeouts()`) and its `set_nonblocking` uses deprecated `PIPE_NOWAIT`, which corrupts mid-stream (proven by the mesh E2E). Blocking reads work on both OSes; the channel supplies the deadline. A read that exceeds the deadline returns `io::ErrorKind::TimedOut`, treated as a **POISONED client** (a late reply could bind the wrong stream id in a later call — gotcha-#1-adjacent): it BUBBLES out of the round → `run_peer_pump` returns Err → `supervise_pump` restarts the pump with a fresh brain client + conn cache + reset WorkerLasts (the §V4 stagger re-primes every leg). NEVER a per-peer retry. An ordinary error (the broker REPLIED with one) stays a per-peer abort + conn drop + redial. **Leak watch (accepted, not fixed):** a restart abandons the old reader thread parked in `read_frame` (split halves share one OS handle → on a named pipe the broker may never signal the disconnect); bounded to ONE thread per actual wedge (the post-restart conn cache is fresh — the dead peer is re-dialed, not re-wedged), diagnosable by the reader's spawn/exit log lines, and fully cured only by the broker-side B-half (DEFERRED.md).
370	- **spt-core mapping:** `Brain::cold_start_pump` (splits the carrier + arms the deadline) / `BrainConn::Split` (the `SendHalf` + reader-thread channel) / `call_deadline` / `read_event_until` / `read_frame_until` (the `recv_timeout` dispatch), `pump::run_peer_pump` (connects in pump mode) + `pump::peer_outcome` (the tier-split), the `request_sync`/`request_update` pull loops (deadline re-armed on progress). The broker-side half — the broker must never make a brain wait unbounded on a QUIC op (bound `net_open_stream`/`send`/`dial` handlers) — is deferred to the next broker-update batch (DEFERRED.md).
371	- **Source:** field diagnosis 2026-06-11 (the 2.2h hfenduleam wedge); doyle ruling A-now / B-deferred. The stall warning (M8 decision 23) was the band-aid; this is the fix.
372
373	### 7.7 A slow/dead/hostile remote VIEWER must never stall the controller, child, or drain  `[REQ-HAZARD-VIEWER-ISOLATION]`
374	<!-- [doc->REQ-HAZARD-VIEWER-ISOLATION] -->
375	- **Failure:** the W2.5 controller/viewer model lets ANY number of read-only `--view` attachers ride one session's broker `OutputLog`. The single drain thread fans each output chunk to every attacher. If a viewer's socket is fanned out with a **blocking** write under the log lock (the controller's authoritative path), one wedged viewer (a slow terminal, a black-holed WAN peer, a hostile non-reader) stalls the drain — freezing the controller's stream and backing up the PTY child. A single watcher must never be able to degrade or hang the driver.
376	- **Invariant:** the drain writes the **controller** on the authoritative blocking bounded path (it alone advances `delivered_through`), but each **viewer** gets an **isolated bounded SPSC queue + a dedicated writer thread**; the drain `try_send`s under the log lock and **evicts** any viewer whose queue is `Full` (fell behind the live stream) or `Disconnected` (its writer died on a dead socket) — the drain thread **never touches a viewer socket**, so no viewer write can backpressure it. A **soft cap** (`MAX_VIEWERS`) bounds the writer-thread count (a viewer attach beyond it is refused). Viewer eviction never perturbs the controller stream, the `delivered_through` cursor, or the child. Ring **replay-at-attach** is owned by the writer thread (not the bounded live queue), so a viewer attaching to a busy session is not spuriously evicted.
377	- **spt-core mapping:** `OutputLog::append` (controller blocking + `viewer_send_evicts` `try_send` fan-out), `OutputLog::add_viewer` (bounded `sync_channel(VIEWER_CHANNEL_DEPTH)` + `viewer_writer` thread), `MAX_VIEWERS` soft cap, `ViewerSink`. Unit: `viewer_overflow_or_disconnect_evicts_never_blocks`. Int: `wedged_viewer_does_not_stall_controller` (a non-reading viewer is evicted while the controller keeps receiving past a 200KB burst).
378	- **Source:** M12 W2.5 controller/viewer model (doyle ruling 2026-06-14, Q1).
379
380	---
381
382	## Conformance checklist (condensed)
383
384	| # | Invariant | spt-core surface |
385	|---|---|---|
386	| 1.1 | Grace wait precedes INIT_SIGNOFF | daemon teardown |
387	| 1.4/4.4 | Deferred rows excluded from event-stream drain | daemon spool drain |
388	| 2.1/5.1 | Stable PID/broker-handle over ephemeral PID | liveness detection |
389	| 2.3 | Handoff argv/IPC version-tolerant (newer brain ↔ older broker) | broker↔brain IPC, self-update |
390	| 2.4 | gen_start = now() on cold-start + handoff | per-instance generation |
391	| 3.1 | Ephemeral perch cleanup on all exit paths | `ring` (RAII guard) |
392	| 3.3 | Echo-commune before INIT_SIGNOFF | daemon psyche loop |
393	| 4.1 | Envelope decode order, `&` last | spt-proto (public, wire-versioned) |
394	| 4.2 | Parser panic-free + tolerant | spt-proto |
395	| 4.3 | Stale registry entries → fallback, never hard-fail | subnet registry resolution |
396	| 4.10 | Silent-node registry rows evicted (heard-map TTL); own rows never decay | registry pump eviction |
397	| 4.6 | Addressable-id charset reserves `:`/`@` delimiters | `spt_proto::id` at creation seams |
398	| 5.2 | tmp-write + atomic-rename + retry (EBUSY) | all state writes, binary swap |
399	| 5.3 | Timeout every harness subprocess | manifest invocations |
400	| 5.4 | Strip UNC prefix on serialized paths | spt-proto path normalization |
401	| 5.5 | ConPTY reader answers DSR (`ESC[6n`) | spt-term broker PTY reader |
402	| 7.7 | A wedged viewer is evicted, never stalls the controller/child/drain | broker `OutputLog` fan-out (controller/viewer) |
403	| 5.6 | Detached long-lived children spawn `bInheritHandles=FALSE` | daemon + shell spawn |
404	| 5.7 | Daemon always unelevated in the invoker's universe (de-elevated spawn + entry guard) | `deelevate` seam, `spawn_detached`, `Daemon::run` |
405	| 5.8 | `CREATE_NO_WINDOW` on every console child of the console-less daemon | `gitrun::run_git`, `kill_shell_pid`, `run_bounded_command`, `shellwake` |
406	| 6.1 | Single path/registry source of truth | storage layout |
407	| 6.4 | Drop files supervisor-owned single-writer | runtime contract |
408	| 6.5 | Direct-write precedence marker (+ node id) | cross-node Psyche sync |
409	| 6.6 | Surfaced conflicts preserve both versions until dominated | context conflict artifacts (ADR-0013) |
410	| 6.7 | Broker + brain are separate processes (brain restart never drops a hosted endpoint at the process level) | daemon process topology, self-update (ADR-0018) |
411	| 6.8 | No irreversible durable-state migration before update ready-promotion (pre-ready writes stay N-1-readable) | auto-rollback / durable-state schema (ADR-0018) |
412	| 6.10 | Phase-significant loop timing is a durable absolute-deadline grid (no per-fire write; update preserves phase, crash resets, one-shot never resets) | durable loop timing / self-update (ADR-0018 Q4) |
413	| 6.11 | Brain respawn execs the applied bytes (canonical exe captured at broker start, not per-spawn current_exe) + promotion bytes-gate (exe_hash == artifact, else rollback) | daemon respawn path / self-update (ADR-0018 Q3) |
414	| 7.1 | Local `api` mutation authenticated to endpoint | api surface / broker IPC |
415	| 7.2 | Idempotent delivery across brain restart | broker↔brain IPC |
416	| 7.3 | Psyche outbound captured + `from=`/target stripped + reply-to-sender / notify-to-own-user | live-Psyche driver / daemon relay (ADR-0012) |
417	| 7.4 | Per-agent pulse/psyche/echo runs off the shared scheduler (no serial blocking across agents) | daemon multi-agent hosting (ADR-0004) |
418	| 7.5 | WAN-inbound origin = QUIC handshake identity from the broker's stream table, never payload bytes | wan receive funnel + every wire-inbound consumer (ADR-0009) |
419	| 7.6 | Pump brain-IPC reads deadline-bounded (30s total-wait); TimedOut → supervised restart, never per-peer retry | `Brain::cold_start_pump` / `BrainConn::Split` (reader-thread + `recv_timeout`), `pump::peer_outcome` |
420

### 7.6 Pump brain-IPC reads must be deadline-bounded (a blocked read wedges the whole pump)  `[REQ-HAZARD-PUMP-IPC-DEADLINE]`
<!-- [doc->REQ-HAZARD-PUMP-IPC-DEADLINE] -->
- **Failure:** the peer pump is a SINGLE thread driving every leg (registry/notif/sync/update) against every peer over ONE brain-IPC client. Its reply reads (`net_open_stream`, `net_stream_send`, `net_dial`, and the sync/update pull `read_event` loops) were `loop { read_event() }` with no deadline. When a peer's QUIC path black-holes, the broker's stream-open/send awaits the dead peer and never sends the reply, so the brain's `read_frame` blocks FOREVER and the pump freezes mid-round. The heartbeat (loop-top) stops and the stall warning fires honestly, but `supervise_pump` cannot rescue it: the supervisor catches a panic / error / clean return, and a BLOCKED thread never returns. Observed twice — 2026-06-07, then a **2.2h wedge on hfenduleam (2026-06-11)** where the broker stayed responsive to `daemon status` while the pump thread sat dead.
- **Invariant:** in PUMP mode the brain carrier is **SPLIT at construction** — a dedicated `pump-ipc-reader` thread does blocking `read_frame` on the `RecvHalf` and forwards each framed result down a channel; the main thread writes on the `SendHalf` and reads with `Receiver::recv_timeout`. Every IPC reply read is bounded by a per-call **total-wait** deadline (`PUMP_PEER_IO_TIMEOUT` = 30s, > any legitimate round-trip, < the 60s QUIC idle; re-armed on stream progress for the streaming pull legs so a healthy long sync is never killed — only a ≥30s silence). **Mechanism note (Windows):** the carrier is a reader-thread + channel, NOT a non-blocking socket + poll — interprocess 2.4.2 on Windows named pipes has no portable read timeout (`set_recv_timeout` → `no_timeouts()`) and its `set_nonblocking` uses deprecated `PIPE_NOWAIT`, which corrupts mid-stream (proven by the mesh E2E). Blocking reads work on both OSes; the channel supplies the deadline. A read that exceeds the deadline returns `io::ErrorKind::TimedOut`, treated as a **POISONED client** (a late reply could bind the wrong stream id in a later call — gotcha-#1-adjacent): it BUBBLES out of the round → `run_peer_pump` returns Err → `supervise_pump` restarts the pump with a fresh brain client + conn cache + reset WorkerLasts (the §V4 stagger re-primes every leg). NEVER a per-peer retry. An ordinary error (the broker REPLIED with one) stays a per-peer abort + conn drop + redial. **Leak watch (accepted, not fixed):** a restart abandons the old reader thread parked in `read_frame` (split halves share one OS handle → on a named pipe the broker may never signal the disconnect); bounded to ONE thread per actual wedge (the post-restart conn cache is fresh — the dead peer is re-dialed, not re-wedged), diagnosable by the reader's spawn/exit log lines, and fully cured only by the broker-side B-half (DEFERRED.md).
- **spt-core mapping:** `Brain::cold_start_pump` (splits the carrier + arms the deadline) / `BrainConn::Split` (the `SendHalf` + reader-thread channel) / `call_deadline` / `read_event_until` / `read_frame_until` (the `recv_timeout` dispatch), `pump::run_peer_pump` (connects in pump mode) + `pump::peer_outcome` (the tier-split), the `request_sync`/`request_update` pull loops (deadline re-armed on progress). The broker-side half — the broker must never make a brain wait unbounded on a QUIC op (bound `net_open_stream`/`send`/`dial` handlers) — is deferred to the next broker-update batch (DEFERRED.md).
- **Source:** field diagnosis 2026-06-11 (the 2.2h hfenduleam wedge); doyle ruling A-now / B-deferred. The stall warning (M8 decision 23) was the band-aid; this is the fix.

### 7.7 A slow/dead/hostile remote VIEWER must never stall the controller, child, or drain  `[REQ-HAZARD-VIEWER-ISOLATION]`
<!-- [doc->REQ-HAZARD-VIEWER-ISOLATION] -->
- **Failure:** the W2.5 controller/viewer model lets ANY number of read-only `--view` attachers ride one session's broker `OutputLog`. The single drain thread fans each output chunk to every attacher. If a viewer's socket is fanned out with a **blocking** write under the log lock (the controller's authoritative path), one wedged viewer (a slow terminal, a black-holed WAN peer, a hostile non-reader) stalls the drain — freezing the controller's stream and backing up the PTY child. A single watcher must never be able to degrade or hang the driver.
- **Invariant:** the drain writes the **controller** on the authoritative blocking bounded path (it alone advances `delivered_through`), but each **viewer** gets an **isolated bounded SPSC queue + a dedicated writer thread**; the drain `try_send`s under the log lock and **evicts** any viewer whose queue is `Full` (fell behind the live stream) or `Disconnected` (its writer died on a dead socket) — the drain thread **never touches a viewer socket**, so no viewer write can backpressure it. A **soft cap** (`MAX_VIEWERS`) bounds the writer-thread count (a viewer attach beyond it is refused). Viewer eviction never perturbs the controller stream, the `delivered_through` cursor, or the child. Ring **replay-at-attach** is owned by the writer thread (not the bounded live queue), so a viewer attaching to a busy session is not spuriously evicted.
- **spt-core mapping:** `OutputLog::append` (controller blocking + `viewer_send_evicts` `try_send` fan-out), `OutputLog::add_viewer` (bounded `sync_channel(VIEWER_CHANNEL_DEPTH)` + `viewer_writer` thread), `MAX_VIEWERS` soft cap, `ViewerSink`. Unit: `viewer_overflow_or_disconnect_evicts_never_blocks`. Int: `wedged_viewer_does_not_stall_controller` (a non-reading viewer is evicted while the controller keeps receiving past a 200KB burst).
- **Source:** M12 W2.5 controller/viewer model (doyle ruling 2026-06-14, Q1).

---

## Conformance checklist (condensed)

| # | Invariant | spt-core surface |
|---|---|---|
| 1.1 | Grace wait precedes INIT_SIGNOFF | daemon teardown |
| 1.4/4.4 | Deferred rows excluded from event-stream drain | daemon spool drain |
| 2.1/5.1 | Stable PID/broker-handle over ephemeral PID | liveness detection |
| 2.3 | Handoff argv/IPC version-tolerant (newer brain ↔ older broker) | broker↔brain IPC, self-update |
| 2.4 | gen_start = now() on cold-start + handoff | per-instance generation |
| 3.1 | Ephemeral perch cleanup on all exit paths | `ring` (RAII guard) |
| 3.3 | Echo-commune before INIT_SIGNOFF | daemon psyche loop |
| 4.1 | Envelope decode order, `&` last | spt-proto (public, wire-versioned) |
| 4.2 | Parser panic-free + tolerant | spt-proto |
| 4.3 | Stale registry entries → fallback, never hard-fail | subnet registry resolution |
| 4.10 | Silent-node registry rows evicted (heard-map TTL); own rows never decay | registry pump eviction |
| 4.6 | Addressable-id charset reserves `:`/`@` delimiters | `spt_proto::id` at creation seams |
| 5.2 | tmp-write + atomic-rename + retry (EBUSY) | all state writes, binary swap |
| 5.3 | Timeout every harness subprocess | manifest invocations |
| 5.4 | Strip UNC prefix on serialized paths | spt-proto path normalization |
| 5.5 | ConPTY reader answers DSR (`ESC[6n`) | spt-term broker PTY reader |
| 7.7 | A wedged viewer is evicted, never stalls the controller/child/drain | broker `OutputLog` fan-out (controller/viewer) |
| 5.6 | Detached long-lived children spawn `bInheritHandles=FALSE` | daemon + shell spawn |
| 5.7 | Daemon always unelevated in the invoker's universe (de-elevated spawn + entry guard) | `deelevate` seam, `spawn_detached`, `Daemon::run` |
| 5.8 | `CREATE_NO_WINDOW` on every console child of the console-less daemon | `gitrun::run_git`, `kill_shell_pid`, `run_bounded_command`, `shellwake` |
| 6.1 | Single path/registry source of truth | storage layout |
| 6.4 | Drop files supervisor-owned single-writer | runtime contract |
| 6.5 | Direct-write precedence marker (+ node id) | cross-node Psyche sync |
| 6.6 | Surfaced conflicts preserve both versions until dominated | context conflict artifacts (ADR-0013) |
| 6.7 | Broker + brain are separate processes (brain restart never drops a hosted endpoint at the process level) | daemon process topology, self-update (ADR-0018) |
| 6.8 | No irreversible durable-state migration before update ready-promotion (pre-ready writes stay N-1-readable) | auto-rollback / durable-state schema (ADR-0018) |
| 6.10 | Phase-significant loop timing is a durable absolute-deadline grid (no per-fire write; update preserves phase, crash resets, one-shot never resets) | durable loop timing / self-update (ADR-0018 Q4) |
| 6.11 | Brain respawn execs the applied bytes (canonical exe captured at broker start, not per-spawn current_exe) + promotion bytes-gate (exe_hash == artifact, else rollback) | daemon respawn path / self-update (ADR-0018 Q3) |
| 7.1 | Local `api` mutation authenticated to endpoint | api surface / broker IPC |
| 7.2 | Idempotent delivery across brain restart | broker↔brain IPC |
| 7.3 | Psyche outbound captured + `from=`/target stripped + reply-to-sender / notify-to-own-user | live-Psyche driver / daemon relay (ADR-0012) |
| 7.4 | Per-agent pulse/psyche/echo runs off the shared scheduler (no serial blocking across agents) | daemon multi-agent hosting (ADR-0004) |
| 7.5 | WAN-inbound origin = QUIC handshake identity from the broker's stream table, never payload bytes | wan receive funnel + every wire-inbound consumer (ADR-0009) |
| 7.6 | Pump brain-IPC reads deadline-bounded (30s total-wait); TimedOut → supervised restart, never per-peer retry | `Brain::cold_start_pump` / `BrainConn::Split` (reader-thread + `recv_timeout`), `pump::peer_outcome` |


Now locate DEFERRED.md's B-half entry.


Bash

ls -la DEFERRED.md docs/DEFERRED.md 2>/dev/null; echo "==="; grep -rn "broker-side\|B-half\|QUIC op\|unbounded\|PUMP-IPC" DEFERRED.md docs/DEFERRED.md 2>/dev/null

Find DEFERRED.md B-half entry


Error: Exit code 2
-rw-r--r-- 1 decid 197609 40653 Jun 16 03:58 docs/DEFERRED.md
===
docs/DEFERRED.md:42:| **Broker-side bound on brain-waiting QUIC ops (the pump-IPC-deadline B-half)**: the brain-side deadline (REQ-HAZARD-PUMP-IPC-DEADLINE, KNOWN-HAZARDS 7.6) stops the single-threaded pump wedging when a peer black-holes, by bounding the pump's brain-IPC reads and escalating a TimedOut to a supervised restart. But the ROOT cause is broker-side: the broker's `net_open_stream` / `net_stream_send` / `net_dial` handlers make the brain wait on a QUIC operation to a dead peer with no bound of their own, so the brain only escapes via its own deadline (a 30s stall + a full pump restart each time). FIX = the broker must never make a brain wait unbounded on a QUIC op — bound those handlers broker-side (a QUIC-level timeout that returns an Error event promptly), so a black-holed peer fails the single `peer_step` cleanly (ordinary per-peer abort + redial) without poisoning the client or forcing a pump restart. The broker is RESIDENT by design (CONTEXT §"broker update frequency: rare"), so this cannot reach the deployed fleet without a broker-update path — hence deferred to the next planned broker-update batch rather than riding the brain-only v0.4.1 roll the A-half ships on. **Added motivation (Windows, KH 7.6):** the A-half's reader-thread carrier leaks ONE `pump-ipc-reader` thread per wedge event — on a supervised restart the old reader is left blocked in `read_frame` holding its `RecvHalf`, and interprocess split halves share one OS handle (closes only when BOTH drop), so on a named pipe the broker may never signal the disconnect and the old reader can park indefinitely (especially while the broker itself is wedged on the dead-peer QUIC await — the very thing B fixes). Bounded (one thread per actual wedge, not per restart round — the post-restart conn cache is fresh, so the dead peer is re-dialed, not re-wedged on a cached dead conn), but B is the real cure: bounding the broker's QUIC await makes the brain's read return promptly, so the reader thread is never left parked | Pump-stall diagnosis 2026-06-11 (doyle ruling A-now / B-deferred) | The brain-side A-half fixes the fleet NOW via the brain-only update machinery (v0.4.1 candidate) and makes the wedge self-heal in 5s instead of 2.2h; B is the deeper correctness fix but only reachable through the rare broker-update path. Mint a REQ (broker-side QUIC-op bound) before building | Next planned broker-update batch (folds in with any other resident-broker change) |


36	| `xtask debug-converge` — automated convergence watcher for debug rollout (the REQ-UPD-6 first-slice item ADR-0016 names): watch every expected debug-pinned reachable lab node until all report the target version applied, else a per-node timeout table (Offline / StagedAwaitingConsent / BlockedByBrokerResources / Rejected{reason}). Activates REQ-UPD-6's `int` stage. Full build plan: `docs/DEBUG-CONVERGE-PLAN.md` | REQ-UPD-6 first slice (2026-06-06) | The first slice ships manual "Apply and observe" (`DEBUG-ROLLOUT.md`); the watcher's one genuinely new surface is a node status-query wire leg (no-fetch `{channel, applied_version, last_outcome}` over the existing update handshake) — assembly otherwise | First multi-node debug rollout where hand-walking nodes is the bottleneck, or M8 update polish |
37	| **Origin-source update bootstrap (`spt update fetch` — pull the latest signed release from GitHub into the existing verify→stage→apply pipeline)**: today update DISCOVERY is **peer-only** (REQ-UPD-1: a daemon pulls offers from roster peers via `request_update`; `propagate.rs`/`peerloop.rs`). There is NO origin fetch, so the **first node in a fleet — or any isolated/solo node — cannot get a new release without a peer that already staged it** (the fleet converges by gossip, but only *after* a maintainer seeds the first node via the release-publish path). The installer DOES fetch from GitHub (`github.com/SaberMage/spt-releases/releases/latest/download/<asset>` + `SHA256SUMS`, `installer/install.sh:57-73`) but only at install time, outside the daemon's signed-release gate, and never re-runs. FIX = a new **origin-fetch leg** that downloads the per-platform artifact + its `<asset>.release.json` (the `SignedRelease` metadata) from the release origin and feeds them through the **SAME** `plan_verified` gate (`verify_metadata` two-key signature/channel/expiry/rollback + `verify_artifact` SHA-256, `update.rs:157`/`release.rs`), then `cache.stage()` — after which the EXISTING consent-notif / `spt update apply` flow is unchanged. So it reuses ~all the machinery; it only adds an HTTPS GET from the origin as an alternate SOURCE to the P2P `request_update` pull. Surface: `spt update fetch [--channel <ch>] [--tag vX.Y.Z]` (default: latest on the node's pinned channel); optionally a daemon-config `origin_check` cadence so a lone node self-bootstraps alongside P2P, then re-propagates to peers normally. **Key design note:** the daemon has **zero HTTP surface today** — all transport is iroh QUIC; `reqwest` is only a transitive dep (via iroh), not reachable. This leg needs a deliberate HTTPS-client decision (add `reqwest`/a minimal client as a direct dep, vs shell out to `curl`/`Invoke-WebRequest` like the installer — fragile). Honor the channel pin (`release-keys.json`) + the monotonic rollback floor; configurable repo (mirror `SPT_INSTALL_REPO`). The two-key signed-release anchor keeps the GitHub transport **untrusted-but-verified** (strictly stronger than the installer's HTTPS+SHA256SUMS-only first fetch). Mint **REQ-UPD-7** (origin-source fetch) first. | Update subsystem v1 (ADR-0016, REQ-UPD family) — P2P-pull only by design | v1 update flow is peer-propagation by design (seed one node, the fleet converges by gossip); origin-fetch is additive and the maintainer seed path covered bootstrap so far. No HTTP client is wired into the daemon yet | First isolated / first-in-fleet node that must update with no peer to pull from; or making `spt update` self-sufficient for solo / single-node users (user flagged the gap 2026-06-08) |
38	| **Test broker socket-bind hardening (CI-flake removal)**: `applyhost` tests' `served_broker` helper does `Broker::bind(name).expect("bind broker")` (`crates/spt-daemon/src/applyhost.rs:293`) where `name` = `spt-daemon-d7b-{pid}-{seq}.sock`. Flakes intermittently on the **kitsubito** self-hosted Linux runner (likely AF_UNIX path collision / stale socket file under the runner's reused workspace — Windows named pipes don't hit it). Cost the v0.3.2 release a full red CI run + a `--failed` rerun (2026-06-09); same code passed clean on rerun → flake, not a logic bug. FIX = make the test bind deterministic: unlink-before-bind, OR abstract-namespace socket (Linux `@`-prefixed, no filesystem entry), OR per-test `TempDir` isolation so each bind has a guaranteed-fresh path. Applies to every `served_broker`/socket-name test helper, not just applyhost | Triaged during v0.3.2 release (2026-06-09) — flake, shipped over it | Non-blocking: a real flake that taxes CI (one red cycle + rerun per hit) but never gates a correct build; hardening is test-infra work, not product. Held to keep the v0.3.2 release moving | **Restoration D1/D7** (`RESTORATION-PLAN.md`) — D1/D7 add real process-spawn tests on the kitsubito runner; fix the bind determinism there rather than fighting flakes through the milestone (M8 is complete) |
39	| **`spt update fetch` single→set shape-upgrade at same version (REQ-UPD-8 transitional trap)**: `cmd_update_fetch` sets the rollback floor = `staged_version` (the monotonic counter), and the gate is a plain `candidate > floor`. A node that fetched+staged version N as a platform **single** (pre-0.3.2 fetch behavior) can never re-fetch version N as a multi-platform **set** — same counter → `Rollback{current:N, candidate:N}`. Bites the **seed-node bootstrap**: the first node to origin-`fetch` a release applies the single for its OWN platform, then can't re-stage the same version as a set → has no Windows/other-platform artifact to SERVE peers over P2P. Surfaced live in the v0.3.2 cross-platform rig verify (2026-06-09): kitsubito staged+applied the v6 single, then its v6-set re-fetch was refused; unblock was a manual `releases/` cache-clear before re-fetch. Counter vs semver is working as designed (counter = trust spine, semver = cosmetic) — the gap is purely shape-blind: the floor compares version only, ignoring single-vs-set. FIX options: (a) allow a same-version re-fetch when staged shape is `single` and an available `set` is strictly-richer (shape-upgrade allowance), or (b) seed-server flow stages the SET before applying its own single, or (c) a `--force`/`--restage` flag. NOTE — only bites the one migration hop onto 0.3.2; once a node is on 0.3.2 with a fresh cache, future releases are vN+1 and stage clean. Mint a REQ before fixing | v0.3.2 cross-platform rig verify (2026-06-09; user-flagged via session watch) | Transitional-only: the workaround (cache-clear before staging the set) is reliable and was used to complete the v0.3.2 fleet roll; a code fix only saves the manual clear on same-version shape upgrades, which won't recur after the 0.3.2 hop | A future release that again changes staged-artifact SHAPE at a same/non-incrementing counter, or making `spt update fetch` self-sufficient for solo seed-nodes (pairs with the origin-source bootstrap row above) |
40	| **On-disk `current_exe` reconcile after auto-rollback (the D6-2 reboot residual)**: readiness-gated auto-rollback (ADR-0018 Q7, restoration D6-2) is **record-driven selection** — after a rollback the broker's supervisor keeps spawning the last-known-good `.old-N` binary, chosen from the durable `RolledBack` record, so the rescue survives a reboot for free **with no file rename at the failure instant** (the brittle path the design rejected). But the bytes on disk at `current_exe` are STILL the quarantined bad version. The supervisor rescues the **brain** (it never execs `current_exe` while a `RolledBack` record stands), but an OS-service / autostart / `ensure_running` relaunch execs `current_exe` **as the broker** — and **if the bad binary panics before broker bind, the node is down until manual intervention** (record-driven selection cannot save a binary that won't boot). FIX = a calm-path reconcile: on the next clean `apply` (or a broker self-reconcile once stable), rename the good bytes over `current_exe` so the on-disk seat matches the running selection. Deliberately NOT a failure-time rename | Restoration D6-2 (ADR-0018 Q7, open-call-3, 2026-06-10) — record-driven selection rescues the brain; the on-disk seat reconcile defers | The reboot-survives-via-record path is correct NOW for the common case (broker boots, supervisor re-selects the good binary); the residual only bites when the broker process itself re-execs the quarantined `current_exe` and that binary won't even bind. Failure-time rename is the brittle path the design rejected; the calm-path reconcile needs the next clean-apply seam | First field rollback where the broker (not just the brain) restarts onto the quarantined on-disk bytes — or the D7 process-survival proof exercising a broker re-exec after rollback |
41	| **Quarantine operator override (`spt update apply --force` — the note-i escape hatch)**: D6-2's quarantine guard refuses re-applying a version that auto-rolled-back on this node (the `RolledBack` record is the marker; clears naturally when N+1 stages). But a version whose failure was **environmental** (a transient boot condition, not bad bytes) leaves the node **permanently allergic** to a genuinely-good version with no escape but staging a newer one. FIX = an explicit operator override (`spt update apply --force`, or a `spt update unquarantine <version>`) that clears the `RolledBack` marker and re-opens the apply for that version — a deliberate, operator-gated act, never automatic | Restoration D6-2 (ADR-0018 Q7, note-i, 2026-06-10) — quarantine ships; the override defers | The natural N+1 clear covers the common "the next release fixes it" path; the override only matters when the SAME version must be retried after an environmental fix. Operator-gated re-apply is a small CLI + record-clear, deferrable to the alarm/adapter era | First node permanently allergic to a genuinely-good version (environmental rollback), or the M8 update-CLI polish |
42	| **Broker-side bound on brain-waiting QUIC ops (the pump-IPC-deadline B-half)**: the brain-side deadline (REQ-HAZARD-PUMP-IPC-DEADLINE, KNOWN-HAZARDS 7.6) stops the single-threaded pump wedging when a peer black-holes, by bounding the pump's brain-IPC reads and escalating a TimedOut to a supervised restart. But the ROOT cause is broker-side: the broker's `net_open_stream` / `net_stream_send` / `net_dial` handlers make the brain wait on a QUIC operation to a dead peer with no bound of their own, so the brain only escapes via its own deadline (a 30s stall + a full pump restart each time). FIX = the broker must never make a brain wait unbounded on a QUIC op — bound those handlers broker-side (a QUIC-level timeout that returns an Error event promptly), so a black-holed peer fails the single `peer_step` cleanly (ordinary per-peer abort + redial) without poisoning the client or forcing a pump restart. The broker is RESIDENT by design (CONTEXT §"broker update frequency: rare"), so this cannot reach the deployed fleet without a broker-update path — hence deferred to the next planned broker-update batch rather than riding the brain-only v0.4.1 roll the A-half ships on. **Added motivation (Windows, KH 7.6):** the A-half's reader-thread carrier leaks ONE `pump-ipc-reader` thread per wedge event — on a supervised restart the old reader is left blocked in `read_frame` holding its `RecvHalf`, and interprocess split halves share one OS handle (closes only when BOTH drop), so on a named pipe the broker may never signal the disconnect and the old reader can park indefinitely (especially while the broker itself is wedged on the dead-peer QUIC await — the very thing B fixes). Bounded (one thread per actual wedge, not per restart round — the post-restart conn cache is fresh, so the dead peer is re-dialed, not re-wedged on a cached dead conn), but B is the real cure: bounding the broker's QUIC await makes the brain's read return promptly, so the reader thread is never left parked | Pump-stall diagnosis 2026-06-11 (doyle ruling A-now / B-deferred) | The brain-side A-half fixes the fleet NOW via the brain-only update machinery (v0.4.1 candidate) and makes the wedge self-heal in 5s instead of 2.2h; B is the deeper correctness fix but only reachable through the rare broker-update path. Mint a REQ (broker-side QUIC-op bound) before building | Next planned broker-update batch (folds in with any other resident-broker change) |

| `xtask debug-converge` — automated convergence watcher for debug rollout (the REQ-UPD-6 first-slice item ADR-0016 names): watch every expected debug-pinned reachable lab node until all report the target version applied, else a per-node timeout table (Offline / StagedAwaitingConsent / BlockedByBrokerResources / Rejected{reason}). Activates REQ-UPD-6's `int` stage. Full build plan: `docs/DEBUG-CONVERGE-PLAN.md` | REQ-UPD-6 first slice (2026-06-06) | The first slice ships manual "Apply and observe" (`DEBUG-ROLLOUT.md`); the watcher's one genuinely new surface is a node status-query wire leg (no-fetch `{channel, applied_version, last_outcome}` over the existing update handshake) — assembly otherwise | First multi-node debug rollout where hand-walking nodes is the bottleneck, or M8 update polish |
| **Origin-source update bootstrap (`spt update fetch` — pull the latest signed release from GitHub into the existing verify→stage→apply pipeline)**: today update DISCOVERY is **peer-only** (REQ-UPD-1: a daemon pulls offers from roster peers via `request_update`; `propagate.rs`/`peerloop.rs`). There is NO origin fetch, so the **first node in a fleet — or any isolated/solo node — cannot get a new release without a peer that already staged it** (the fleet converges by gossip, but only *after* a maintainer seeds the first node via the release-publish path). The installer DOES fetch from GitHub (`github.com/SaberMage/spt-releases/releases/latest/download/<asset>` + `SHA256SUMS`, `installer/install.sh:57-73`) but only at install time, outside the daemon's signed-release gate, and never re-runs. FIX = a new **origin-fetch leg** that downloads the per-platform artifact + its `<asset>.release.json` (the `SignedRelease` metadata) from the release origin and feeds them through the **SAME** `plan_verified` gate (`verify_metadata` two-key signature/channel/expiry/rollback + `verify_artifact` SHA-256, `update.rs:157`/`release.rs`), then `cache.stage()` — after which the EXISTING consent-notif / `spt update apply` flow is unchanged. So it reuses ~all the machinery; it only adds an HTTPS GET from the origin as an alternate SOURCE to the P2P `request_update` pull. Surface: `spt update fetch [--channel <ch>] [--tag vX.Y.Z]` (default: latest on the node's pinned channel); optionally a daemon-config `origin_check` cadence so a lone node self-bootstraps alongside P2P, then re-propagates to peers normally. **Key design note:** the daemon has **zero HTTP surface today** — all transport is iroh QUIC; `reqwest` is only a transitive dep (via iroh), not reachable. This leg needs a deliberate HTTPS-client decision (add `reqwest`/a minimal client as a direct dep, vs shell out to `curl`/`Invoke-WebRequest` like the installer — fragile). Honor the channel pin (`release-keys.json`) + the monotonic rollback floor; configurable repo (mirror `SPT_INSTALL_REPO`). The two-key signed-release anchor keeps the GitHub transport **untrusted-but-verified** (strictly stronger than the installer's HTTPS+SHA256SUMS-only first fetch). Mint **REQ-UPD-7** (origin-source fetch) first. | Update subsystem v1 (ADR-0016, REQ-UPD family) — P2P-pull only by design | v1 update flow is peer-propagation by design (seed one node, the fleet converges by gossip); origin-fetch is additive and the maintainer seed path covered bootstrap so far. No HTTP client is wired into the daemon yet | First isolated / first-in-fleet node that must update with no peer to pull from; or making `spt update` self-sufficient for solo / single-node users (user flagged the gap 2026-06-08) |
| **Test broker socket-bind hardening (CI-flake removal)**: `applyhost` tests' `served_broker` helper does `Broker::bind(name).expect("bind broker")` (`crates/spt-daemon/src/applyhost.rs:293`) where `name` = `spt-daemon-d7b-{pid}-{seq}.sock`. Flakes intermittently on the **kitsubito** self-hosted Linux runner (likely AF_UNIX path collision / stale socket file under the runner's reused workspace — Windows named pipes don't hit it). Cost the v0.3.2 release a full red CI run + a `--failed` rerun (2026-06-09); same code passed clean on rerun → flake, not a logic bug. FIX = make the test bind deterministic: unlink-before-bind, OR abstract-namespace socket (Linux `@`-prefixed, no filesystem entry), OR per-test `TempDir` isolation so each bind has a guaranteed-fresh path. Applies to every `served_broker`/socket-name test helper, not just applyhost | Triaged during v0.3.2 release (2026-06-09) — flake, shipped over it | Non-blocking: a real flake that taxes CI (one red cycle + rerun per hit) but never gates a correct build; hardening is test-infra work, not product. Held to keep the v0.3.2 release moving | **Restoration D1/D7** (`RESTORATION-PLAN.md`) — D1/D7 add real process-spawn tests on the kitsubito runner; fix the bind determinism there rather than fighting flakes through the milestone (M8 is complete) |
| **`spt update fetch` single→set shape-upgrade at same version (REQ-UPD-8 transitional trap)**: `cmd_update_fetch` sets the rollback floor = `staged_version` (the monotonic counter), and the gate is a plain `candidate > floor`. A node that fetched+staged version N as a platform **single** (pre-0.3.2 fetch behavior) can never re-fetch version N as a multi-platform **set** — same counter → `Rollback{current:N, candidate:N}`. Bites the **seed-node bootstrap**: the first node to origin-`fetch` a release applies the single for its OWN platform, then can't re-stage the same version as a set → has no Windows/other-platform artifact to SERVE peers over P2P. Surfaced live in the v0.3.2 cross-platform rig verify (2026-06-09): kitsubito staged+applied the v6 single, then its v6-set re-fetch was refused; unblock was a manual `releases/` cache-clear before re-fetch. Counter vs semver is working as designed (counter = trust spine, semver = cosmetic) — the gap is purely shape-blind: the floor compares version only, ignoring single-vs-set. FIX options: (a) allow a same-version re-fetch when staged shape is `single` and an available `set` is strictly-richer (shape-upgrade allowance), or (b) seed-server flow stages the SET before applying its own single, or (c) a `--force`/`--restage` flag. NOTE — only bites the one migration hop onto 0.3.2; once a node is on 0.3.2 with a fresh cache, future releases are vN+1 and stage clean. Mint a REQ before fixing | v0.3.2 cross-platform rig verify (2026-06-09; user-flagged via session watch) | Transitional-only: the workaround (cache-clear before staging the set) is reliable and was used to complete the v0.3.2 fleet roll; a code fix only saves the manual clear on same-version shape upgrades, which won't recur after the 0.3.2 hop | A future release that again changes staged-artifact SHAPE at a same/non-incrementing counter, or making `spt update fetch` self-sufficient for solo seed-nodes (pairs with the origin-source bootstrap row above) |
| **On-disk `current_exe` reconcile after auto-rollback (the D6-2 reboot residual)**: readiness-gated auto-rollback (ADR-0018 Q7, restoration D6-2) is **record-driven selection** — after a rollback the broker's supervisor keeps spawning the last-known-good `.old-N` binary, chosen from the durable `RolledBack` record, so the rescue survives a reboot for free **with no file rename at the failure instant** (the brittle path the design rejected). But the bytes on disk at `current_exe` are STILL the quarantined bad version. The supervisor rescues the **brain** (it never execs `current_exe` while a `RolledBack` record stands), but an OS-service / autostart / `ensure_running` relaunch execs `current_exe` **as the broker** — and **if the bad binary panics before broker bind, the node is down until manual intervention** (record-driven selection cannot save a binary that won't boot). FIX = a calm-path reconcile: on the next clean `apply` (or a broker self-reconcile once stable), rename the good bytes over `current_exe` so the on-disk seat matches the running selection. Deliberately NOT a failure-time rename | Restoration D6-2 (ADR-0018 Q7, open-call-3, 2026-06-10) — record-driven selection rescues the brain; the on-disk seat reconcile defers | The reboot-survives-via-record path is correct NOW for the common case (broker boots, supervisor re-selects the good binary); the residual only bites when the broker process itself re-execs the quarantined `current_exe` and that binary won't even bind. Failure-time rename is the brittle path the design rejected; the calm-path reconcile needs the next clean-apply seam | First field rollback where the broker (not just the brain) restarts onto the quarantined on-disk bytes — or the D7 process-survival proof exercising a broker re-exec after rollback |
| **Quarantine operator override (`spt update apply --force` — the note-i escape hatch)**: D6-2's quarantine guard refuses re-applying a version that auto-rolled-back on this node (the `RolledBack` record is the marker; clears naturally when N+1 stages). But a version whose failure was **environmental** (a transient boot condition, not bad bytes) leaves the node **permanently allergic** to a genuinely-good version with no escape but staging a newer one. FIX = an explicit operator override (`spt update apply --force`, or a `spt update unquarantine <version>`) that clears the `RolledBack` marker and re-opens the apply for that version — a deliberate, operator-gated act, never automatic | Restoration D6-2 (ADR-0018 Q7, note-i, 2026-06-10) — quarantine ships; the override defers | The natural N+1 clear covers the common "the next release fixes it" path; the override only matters when the SAME version must be retried after an environmental fix. Operator-gated re-apply is a small CLI + record-clear, deferrable to the alarm/adapter era | First node permanently allergic to a genuinely-good version (environmental rollback), or the M8 update-CLI polish |
| **Broker-side bound on brain-waiting QUIC ops (the pump-IPC-deadline B-half)**: the brain-side deadline (REQ-HAZARD-PUMP-IPC-DEADLINE, KNOWN-HAZARDS 7.6) stops the single-threaded pump wedging when a peer black-holes, by bounding the pump's brain-IPC reads and escalating a TimedOut to a supervised restart. But the ROOT cause is broker-side: the broker's `net_open_stream` / `net_stream_send` / `net_dial` handlers make the brain wait on a QUIC operation to a dead peer with no bound of their own, so the brain only escapes via its own deadline (a 30s stall + a full pump restart each time). FIX = the broker must never make a brain wait unbounded on a QUIC op — bound those handlers broker-side (a QUIC-level timeout that returns an Error event promptly), so a black-holed peer fails the single `peer_step` cleanly (ordinary per-peer abort + redial) without poisoning the client or forcing a pump restart. The broker is RESIDENT by design (CONTEXT §"broker update frequency: rare"), so this cannot reach the deployed fleet without a broker-update path — hence deferred to the next planned broker-update batch rather than riding the brain-only v0.4.1 roll the A-half ships on. **Added motivation (Windows, KH 7.6):** the A-half's reader-thread carrier leaks ONE `pump-ipc-reader` thread per wedge event — on a supervised restart the old reader is left blocked in `read_frame` holding its `RecvHalf`, and interprocess split halves share one OS handle (closes only when BOTH drop), so on a named pipe the broker may never signal the disconnect and the old reader can park indefinitely (especially while the broker itself is wedged on the dead-peer QUIC await — the very thing B fixes). Bounded (one thread per actual wedge, not per restart round — the post-restart conn cache is fresh, so the dead peer is re-dialed, not re-wedged on a cached dead conn), but B is the real cure: bounding the broker's QUIC await makes the brain's read return promptly, so the reader thread is never left parked | Pump-stall diagnosis 2026-06-11 (doyle ruling A-now / B-deferred) | The brain-side A-half fixes the fleet NOW via the brain-only update machinery (v0.4.1 candidate) and makes the wedge self-heal in 5s instead of 2.2h; B is the deeper correctness fix but only reachable through the rare broker-update path. Mint a REQ (broker-side QUIC-op bound) before building | Next planned broker-update batch (folds in with any other resident-broker change) |


All grounding complete. Now implement — **commit 1: mint the REQ** (registry-first, `required_stages = []`).


Edit

required_stages = ["doc", "impl", "unit"]  # activated 2026-06-11: deployed v0.4.0 pump wedged 2.2h on hfenduleam — net_open_stream's unbounded `loop { read_event }` blocked on a peer conn the broker never replied for; the single-threaded pump froze and supervise_pump cannot rescue a BLOCKED thread (it only catches panic/error/return). Surfaced twice (2026-06-07 + this 2.2h wedge); the stall-warning was the band-aid, not the fix. fix = Brain pump-mode carrier SPLIT at construction (Brain::cold_start_pump → BrainConn::Split: a `pump-ipc-reader` thread does blocking read_frame on the RecvHalf → channel; main thread writes the SendHalf and reads with Receiver::recv_timeout for a per-call total-wait deadline; deadline re-armed on stream progress for the pull legs). The reader-thread+channel mechanism (NOT a non-blocking socket + poll) is mandatory because interprocess 2.4.2 on Windows named pipes has no portable read timeout and its set_nonblocking corrupts mid-stream (mesh E2E proven). A TimedOut bubbles out of run_peer_pump → supervised restart (fresh brain client + conn cache + WorkerLasts re-prime, the V4 stagger, idempotent). The broker-side bound (broker must never make a brain wait unbounded on a QUIC op) is deferred to the next broker-update batch (DEFERRED.md). doc = KNOWN-HAZARDS entry; unit = (a) a never-replying broker over a real socket → Brain::cold_start_pump → net_open_stream returns TimedOut (never blocks) [tests/pumpdeadline.rs], (b) the recovery tier-split (TimedOut → bubble/Err = supervised restart; ordinary Err → per-peer abort only)

required_stages = ["doc", "impl", "unit"]  # activated 2026-06-11: deployed v0.4.0 pump wedged 2.2h on hfenduleam — net_open_stream's unbounded `loop { read_event }` blocked on a peer conn the broker never replied for; the single-threaded pump froze and supervise_pump cannot rescue a BLOCKED thread (it only catches panic/error/return). Surfaced twice (2026-06-07 + this 2.2h wedge); the stall-warning was the band-aid, not the fix. fix = Brain pump-mode carrier SPLIT at construction (Brain::cold_start_pump → BrainConn::Split: a `pump-ipc-reader` thread does blocking read_frame on the RecvHalf → channel; main thread writes the SendHalf and reads with Receiver::recv_timeout for a per-call total-wait deadline; deadline re-armed on stream progress for the pull legs). The reader-thread+channel mechanism (NOT a non-blocking socket + poll) is mandatory because interprocess 2.4.2 on Windows named pipes has no portable read timeout and its set_nonblocking corrupts mid-stream (mesh E2E proven). A TimedOut bubbles out of run_peer_pump → supervised restart (fresh brain client + conn cache + WorkerLasts re-prime, the V4 stagger, idempotent). The broker-side bound (broker must never make a brain wait unbounded on a QUIC op) is deferred to the next broker-update batch (DEFERRED.md). doc = KNOWN-HAZARDS entry; unit = (a) a never-replying broker over a real socket → Brain::cold_start_pump → net_open_stream returns TimedOut (never blocks) [tests/pumpdeadline.rs], (b) the recovery tier-split (TimedOut → bubble/Err = supervised restart; ordinary Err → per-peer abort only)
# --- Added 2026-06-16 (v0.8.3): the pump-IPC-deadline B-half — the broker-side complement of REQ-HAZARD-PUMP-IPC-DEADLINE (DEFERRED.md:42 resolved); rule 3 register-before-satisfy, rule 5 activate per wave ---
[[requirements]]
id = "REQ-HAZARD-BROKER-QUIC-DEADLINE"
title = "The broker bounds every brain-waiting QUIC op (dial / open_stream / send_stream) so a black-holed or dead peer fails PROMPTLY with an ORDINARY error the broker REPLIES, never an unbounded await. The bound (< the brain's 30s PUMP_PEER_IO_TIMEOUT so the BROKER fires first) surfaces to the pump as a normal broker error reply → peer_outcome's non-TimedOut arm → drop conn + redial next tick, the round CONTINUES and the heartbeat keeps advancing — it must NEVER manifest as the brain's own read-deadline (the A-half poison → supervised-restart path REQ-HAZARD-PUMP-IPC-DEADLINE guards). Exactly-once is preserved: a timed-out journaled op fails INSIDE its apply_once closure so no phantom conn_id/stream_id is recorded and a fresh tick re-dials cleanly. The happy path is unchanged (a live peer completes with zero added latency; the bound only bites a non-responsive peer). This is the ROOT-cause cure for the 2.2h hfenduleam pump wedge — a dead roster peer whose QUIC path the broker awaited unbounded — recurring on hfenduleam 2026-06-16."
required_stages = []  # registry-first mint (rule 3); activated per wave in v0.8.3 (doc = KNOWN-HAZARDS 7.8 + 7.6 B-half-fixed reword + DEFERRED.md:42 resolved; impl = NetHost::bounded_block_on wrapping dial/open_stream/send_stream's QUIC await sites under a quic_op_timeout (default 10s, test-overridable via set_quic_op_timeout); unit = the bound wrapper returns a prompt non-TimedOut error on a never-completing op + a fast op is untouched [nethost.rs]; int = a black-holing peer (membership-mismatch hang) → the broker REPLIES an ordinary error within the bound (not a wedge) + exactly-once-on-timeout (journal not applied, conn table empty) [tests/netbroker.rs], and the pump round survives a dead peer — heartbeat monotonic-advances, no PEER_PUMP_RESTART [tests/pump.rs])


The file C:\Users\decid\Documents\projects\spt-core\traceable-reqs.toml has been updated successfully. (file state is current in your context — no need to Read it back)

required_stages = ["doc", "impl", "unit"]  # activated 2026-06-11: deployed v0.4.0 pump wedged 2.2h on hfenduleam — net_open_stream's unbounded `loop { read_event }` blocked on a peer conn the broker never replied for; the single-threaded pump froze and supervise_pump cannot rescue a BLOCKED thread (it only catches panic/error/return). Surfaced twice (2026-06-07 + this 2.2h wedge); the stall-warning was the band-aid, not the fix. fix = Brain pump-mode carrier SPLIT at construction (Brain::cold_start_pump → BrainConn::Split: a `pump-ipc-reader` thread does blocking read_frame on the RecvHalf → channel; main thread writes the SendHalf and reads with Receiver::recv_timeout for a per-call total-wait deadline; deadline re-armed on stream progress for the pull legs). The reader-thread+channel mechanism (NOT a non-blocking socket + poll) is mandatory because interprocess 2.4.2 on Windows named pipes has no portable read timeout and its set_nonblocking corrupts mid-stream (mesh E2E proven). A TimedOut bubbles out of run_peer_pump → supervised restart (fresh brain client + conn cache + WorkerLasts re-prime, the V4 stagger, idempotent). The broker-side bound (broker must never make a brain wait unbounded on a QUIC op) is deferred to the next broker-update batch (DEFERRED.md). doc = KNOWN-HAZARDS entry; unit = (a) a never-replying broker over a real socket → Brain::cold_start_pump → net_open_stream returns TimedOut (never blocks) [tests/pumpdeadline.rs], (b) the recovery tier-split (TimedOut → bubble/Err = supervised restart; ordinary Err → per-peer abort only)
# --- Added 2026-06-16 (v0.8.3): the pump-IPC-deadline B-half — the broker-side complement of REQ-HAZARD-PUMP-IPC-DEADLINE (DEFERRED.md:42 resolved); rule 3 register-before-satisfy, rule 5 activate per wave ---
[[requirements]]
id = "REQ-HAZARD-BROKER-QUIC-DEADLINE"
title = "The broker bounds every brain-waiting QUIC op (dial / open_stream / send_stream) so a black-holed or dead peer fails PROMPTLY with an ORDINARY error the broker REPLIES, never an unbounded await. The bound (< the brain's 30s PUMP_PEER_IO_TIMEOUT so the BROKER fires first) surfaces to the pump as a normal broker error reply → peer_outcome's non-TimedOut arm → drop conn + redial next tick, the round CONTINUES and the heartbeat keeps advancing — it must NEVER manifest as the brain's own read-deadline (the A-half poison → supervised-restart path REQ-HAZARD-PUMP-IPC-DEADLINE guards). Exactly-once is preserved: a timed-out journaled op fails INSIDE its apply_once closure so no phantom conn_id/stream_id is recorded and a fresh tick re-dials cleanly. The happy path is unchanged (a live peer completes with zero added latency; the bound only bites a non-responsive peer). This is the ROOT-cause cure for the 2.2h hfenduleam pump wedge — a dead roster peer whose QUIC path the broker awaited unbounded — recurring on hfenduleam 2026-06-16."
required_stages = []  # registry-first mint (rule 3); activated per wave in v0.8.3 (doc = KNOWN-HAZARDS 7.8 + 7.6 B-half-fixed reword + DEFERRED.md:42 resolved; impl = NetHost::bounded_block_on wrapping dial/open_stream/send_stream's QUIC await sites under a quic_op_timeout (default 10s, test-overridable via set_quic_op_timeout); unit = the bound wrapper returns a prompt non-TimedOut error on a never-completing op + a fast op is untouched [nethost.rs]; int = a black-holing peer (membership-mismatch hang) → the broker REPLIES an ordinary error within the bound (not a wedge) + exactly-once-on-timeout (journal not applied, conn table empty) [tests/netbroker.rs], and the pump round survives a dead peer — heartbeat monotonic-advances, no PEER_PUMP_RESTART [tests/pump.rs])

title = "The single-threaded peer pump's brain-IPC reads are deadline-bounded (PUMP_PEER_IO_TIMEOUT, total-wait per call); a TimedOut read POISONS the client and escalates to a SUPERVISED RESTART, never a per-peer retry — a black-holed peer must never wedge the whole pump"

+# --- Added 2026-06-16 (v0.8.3): the pump-IPC-deadline B-half — the broker-side complement of REQ-HAZARD-PUMP-IPC-DEADLINE (DEFERRED.md:42 resolved); rule 3 register-before-satisfy, rule 5 activate per wave ---

+title = "The broker bounds every brain-waiting QUIC op (dial / open_stream / send_stream) so a black-holed or dead peer fails PROMPTLY with an ORDINARY error the broker REPLIES, never an unbounded await. The bound (< the brain's 30s PUMP_PEER_IO_TIMEOUT so the BROKER fires first) surfaces to the pump as a normal broker error reply → peer_outcome's non-TimedOut arm → drop conn + redial next tick, the round CONTINUES and the heartbeat keeps advancing — it must NEVER manifest as the brain's own read-deadline (the A-half poison → supervised-restart path REQ-HAZARD-PUMP-IPC-DEADLINE guards). Exactly-once is preserved: a timed-out journaled op fails INSIDE its apply_once closure so no phantom conn_id/stream_id is recorded and a fresh tick re-dials cleanly. The happy path is unchanged (a live peer completes with zero added latency; the bound only bites a non-responsive peer). This is the ROOT-cause cure for the 2.2h hfenduleam pump wedge — a dead roster peer whose QUIC path the broker awaited unbounded — recurring on hfenduleam 2026-06-16."

+required_stages = []  # registry-first mint (rule 3); activated per wave in v0.8.3 (doc = KNOWN-HAZARDS 7.8 + 7.6 B-half-fixed reword + DEFERRED.md:42 resolved; impl = NetHost::bounded_block_on wrapping dial/open_stream/send_stream's QUIC await sites under a quic_op_timeout (default 10s, test-overridable via set_quic_op_timeout); unit = the bound wrapper returns a prompt non-TimedOut error on a never-completing op + a fast op is untouched [nethost.rs]; int = a black-holing peer (membership-mismatch hang) → the broker REPLIES an ordinary error within the bound (not a wedge) + exactly-once-on-timeout (journal not applied, conn table empty) [tests/netbroker.rs], and the pump round survives a dead peer — heartbeat monotonic-advances, no PEER_PUMP_RESTART [tests/pump.rs])

title = "Elevation guidance on Unix names the binary's ABSOLUTE path under sudo (a user-local install ~/.local/bin · ~/.cargo/bin is not on sudo's secure_path, so bare `sudo spt` dies 'command not found'); gated commands auto-elevate on an interactive TTY, else print the runnable hint (5.10)"

required_stages = ["impl", "unit"]  # activated 2026-06-07 field report (KITSUBITO): `spt subnet create` refused unelevated, user ran `sudo spt …` → `sudo: spt: command not found` (spt in ~/.local/bin, off secure_path). fix = elevation::rerun_command emits `sudo <abs-exe> <args>` (shell-quoted) + try_auto_elevate re-execs under sudo on interactive Unix; main.rs drops the confusing DEELEVATED line. unit = absolute-path-not-bare-name + shell-quoting + windows-none + should_auto_elevate truth table (the sudo exec itself needs a real TTY → manual/kitsubito verified)


Commit 1 — the registry-first mint.


Bash

git add traceable-reqs.toml && git commit -q -m "v0.8.3 base: mint REQ-HAZARD-BROKER-QUIC-DEADLINE (registry-first)

The pump-IPC-deadline B-half (DEFERRED.md:42): the broker-side complement
of REQ-HAZARD-PUMP-IPC-DEADLINE. Minted required_stages=[] (rule 3,
register-before-satisfy); stages activate per wave as evidence lands.

Co-authored by: todlando" && git log --oneline -1

Commit REQ mint


69af744 v0.8.3 base: mint REQ-HAZARD-BROKER-QUIC-DEADLINE (registry-first)


Now the impl. Add imports + the bound const.


Edit

use std::collections::{HashMap, HashSet, VecDeque};
use std::io;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};

use std::collections::{HashMap, HashSet, VecDeque};
use std::io;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;


The file C:\Users\decid\Documents\projects\spt-core\crates\spt-daemon\src\nethost.rs has been updated successfully. (file state is current in your context — no need to Read it back)

use std::collections::{HashMap, HashSet, VecDeque};
use std::io;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;

use tokio::io::{AsyncReadExt, AsyncWriteExt, DuplexStream, ReadHalf, WriteHalf};


pub const NET_EFFECT_SESSION: u64 = 0;

/// Broker-side QUIC-op deadline (REQ-HAZARD-BROKER-QUIC-DEADLINE), in ms. Each
/// brain-waiting QUIC await — `dial` / `open_stream` / `send_stream` — is bounded
/// by this so a black-holed or dead peer fails PROMPTLY with an ORDINARY error
/// the broker replies, never an unbounded await that wedges the single-threaded
/// pump (the B-half of REQ-HAZARD-PUMP-IPC-DEADLINE; the 2.2h hfenduleam wedge).
/// Chosen at **10s**: comfortably above any legitimate LAN/relay connect +
/// seed-proof round-trip (sub-second typical, a few seconds over a slow relay),
/// and well below the brain's 30s `PUMP_PEER_IO_TIMEOUT` so the BROKER always
/// fires FIRST — the timeout then surfaces to the pump as a normal broker error
/// reply (`peer_outcome`'s ordinary per-peer arm → drop conn + redial, round
/// continues, heartbeat advances), NOT the brain's own read-deadline (the A-half
/// poison → supervised-restart path). The 20s margin below 30s covers the error
/// frame's IPC transit + the brain's read scheduling. Test-overridable via
/// [`NetHost::set_quic_op_timeout`] so the hermetic black-hole tests fail fast.
const BROKER_QUIC_OP_TIMEOUT_MS: u64 = 10_000;


pub const NET_EFFECT_SESSION: u64 = 0;

/// Broker-side QUIC-op deadline (REQ-HAZARD-BROKER-QUIC-DEADLINE), in ms. Each
/// brain-waiting QUIC await — `dial` / `open_stream` / `send_stream` — is bounded
/// by this so a black-holed or dead peer fails PROMPTLY with an ORDINARY error
/// the broker replies, never an unbounded await that wedges the single-threaded
/// pump (the B-half of REQ-HAZARD-PUMP-IPC-DEADLINE; the 2.2h hfenduleam wedge).
/// Chosen at **10s**: comfortably above any legitimate LAN/relay connect +
/// seed-proof round-trip (sub-second typical, a few seconds over a slow relay),
/// and well below the brain's 30s `PUMP_PEER_IO_TIMEOUT` so the BROKER always
/// fires FIRST — the timeout then surfaces to the pump as a normal broker error
/// reply (`peer_outcome`'s ordinary per-peer arm → drop conn + redial, round
/// continues, heartbeat advances), NOT the brain's own read-deadline (the A-half
/// poison → supervised-restart path). The 20s margin below 30s covers the error
/// frame's IPC transit + the brain's read scheduling. Test-overridable via
/// [`NetHost::set_quic_op_timeout`] so the hermetic black-hole tests fail fast.
const BROKER_QUIC_OP_TIMEOUT_MS: u64 = 10_000;

/// (`Broker::next_id` starts at 1), so `0` can never collide with a real session.

+/// Broker-side QUIC-op deadline (REQ-HAZARD-BROKER-QUIC-DEADLINE), in ms. Each

+/// brain-waiting QUIC await — `dial` / `open_stream` / `send_stream` — is bounded

+/// by this so a black-holed or dead peer fails PROMPTLY with an ORDINARY error

+/// the broker replies, never an unbounded await that wedges the single-threaded

+/// pump (the B-half of REQ-HAZARD-PUMP-IPC-DEADLINE; the 2.2h hfenduleam wedge).

+/// Chosen at **10s**: comfortably above any legitimate LAN/relay connect +

+/// seed-proof round-trip (sub-second typical, a few seconds over a slow relay),

+/// and well below the brain's 30s `PUMP_PEER_IO_TIMEOUT` so the BROKER always

+/// fires FIRST — the timeout then surfaces to the pump as a normal broker error

+/// reply (`peer_outcome`'s ordinary per-peer arm → drop conn + redial, round

+/// continues, heartbeat advances), NOT the brain's own read-deadline (the A-half

+/// poison → supervised-restart path). The 20s margin below 30s covers the error

+/// frame's IPC transit + the brain's read scheduling. Test-overridable via

+/// [`NetHost::set_quic_op_timeout`] so the hermetic black-hole tests fail fast.

+const BROKER_QUIC_OP_TIMEOUT_MS: u64 = 10_000;

/// Everything needed to stand the broker's endpoint up: the node's identity

/// (the endpoint *is* the node's name — EndpointId == pubkey), the relay stance

/// (from `daemon.json` in production; [`RelayPolicy::Disabled`] in hermetic


Add the field to the struct + initialize in `start()`.


    /// Serializes [`Self::open_loopback_pair`] (M11-W3) so the peer-row discovery
    /// (a before/after diff on the loopback conn's inbound rows) sees exactly the
    /// one row this open minted — concurrent pair-opens on the shared loopback conn
    /// would otherwise race the diff.
    loopback_pair_lock: Mutex<()>,
}

    /// Serializes [`Self::open_loopback_pair`] (M11-W3) so the peer-row discovery
    /// (a before/after diff on the loopback conn's inbound rows) sees exactly the
    /// one row this open minted — concurrent pair-opens on the shared loopback conn
    /// would otherwise race the diff.
    loopback_pair_lock: Mutex<()>,
    /// The broker-side QUIC-op deadline in ms (REQ-HAZARD-BROKER-QUIC-DEADLINE),
    /// default [`BROKER_QUIC_OP_TIMEOUT_MS`]; the hermetic black-hole tests lower
    /// it via [`Self::set_quic_op_timeout`]. Interior-mutable so the setter takes
    /// `&self`, like the live host's other post-`start` knobs (`set_roster_exchange`).
    quic_op_timeout_ms: AtomicU64,
}


    /// Serializes [`Self::open_loopback_pair`] (M11-W3) so the peer-row discovery
    /// (a before/after diff on the loopback conn's inbound rows) sees exactly the
    /// one row this open minted — concurrent pair-opens on the shared loopback conn
    /// would otherwise race the diff.
    loopback_pair_lock: Mutex<()>,
}

    /// Serializes [`Self::open_loopback_pair`] (M11-W3) so the peer-row discovery
    /// (a before/after diff on the loopback conn's inbound rows) sees exactly the
    /// one row this open minted — concurrent pair-opens on the shared loopback conn
    /// would otherwise race the diff.
    loopback_pair_lock: Mutex<()>,
    /// The broker-side QUIC-op deadline in ms (REQ-HAZARD-BROKER-QUIC-DEADLINE),
    /// default [`BROKER_QUIC_OP_TIMEOUT_MS`]; the hermetic black-hole tests lower
    /// it via [`Self::set_quic_op_timeout`]. Interior-mutable so the setter takes
    /// `&self`, like the live host's other post-`start` knobs (`set_roster_exchange`).
    quic_op_timeout_ms: AtomicU64,
}

+    /// The broker-side QUIC-op deadline in ms (REQ-HAZARD-BROKER-QUIC-DEADLINE),

+    /// default [`BROKER_QUIC_OP_TIMEOUT_MS`]; the hermetic black-hole tests lower

+    /// it via [`Self::set_quic_op_timeout`]. Interior-mutable so the setter takes

+    /// `&self`, like the live host's other post-`start` knobs (`set_roster_exchange`).


            loopback_conn: Mutex::new(None),
            loopback_pair_lock: Mutex::new(()),
        })

            loopback_conn: Mutex::new(None),
            loopback_pair_lock: Mutex::new(()),
            quic_op_timeout_ms: AtomicU64::new(BROKER_QUIC_OP_TIMEOUT_MS),
        })


            loopback_conn: Mutex::new(None),
            loopback_pair_lock: Mutex::new(()),
        })

            loopback_conn: Mutex::new(None),
            loopback_pair_lock: Mutex::new(()),
            quic_op_timeout_ms: AtomicU64::new(BROKER_QUIC_OP_TIMEOUT_MS),
        })

+            quic_op_timeout_ms: AtomicU64::new(BROKER_QUIC_OP_TIMEOUT_MS),


Add the `bounded_block_on` helper + getter/setter, right after `set_roster_exchange` (the sibling seam).


    pub fn set_roster_exchange(&self, exchange: RosterExchange) {
        *self.roster.lock().unwrap() = Some(exchange);
    }

    pub fn set_roster_exchange(&self, exchange: RosterExchange) {
        *self.roster.lock().unwrap() = Some(exchange);
    }

    /// The broker-side QUIC-op deadline currently in force (default
    /// [`BROKER_QUIC_OP_TIMEOUT_MS`]).
    fn quic_op_timeout(&self) -> Duration {
        Duration::from_millis(self.quic_op_timeout_ms.load(Ordering::Relaxed))
    }

    /// Override the broker-side QUIC-op deadline (REQ-HAZARD-BROKER-QUIC-DEADLINE).
    /// Production keeps the [`BROKER_QUIC_OP_TIMEOUT_MS`] default; the hermetic
    /// black-hole tests set a short bound so a dead-peer await fails fast instead
    /// of pinning the test for ~10s. Off [`NetConfig`] by design (mirrors
    /// [`Self::set_roster_exchange`]) — the many hermetic test configs need no change.
    // [impl->REQ-HAZARD-BROKER-QUIC-DEADLINE]
    pub fn set_quic_op_timeout(&self, dur: Duration) {
        self.quic_op_timeout_ms
            .store(dur.as_millis() as u64, Ordering::Relaxed);
    }

    /// Run `fut` on the host runtime under the broker-side QUIC-op deadline
    /// (REQ-HAZARD-BROKER-QUIC-DEADLINE) — the B-half of REQ-HAZARD-PUMP-IPC-
    /// DEADLINE. A brain-waiting QUIC await against a black-holed/dead peer must
    /// fail PROMPTLY with an ORDINARY error the broker replies, never the
    /// unbounded await that wedged the single-threaded pump for 2.2h. On elapse
    /// the future is DROPPED — cancelling the in-flight connect / stream op, so
    /// nothing is half-registered — and a non-`TimedOut` `io::Error` is returned.
    /// At the broker boundary that error is stringified into a normal error reply
    /// frame, so the brain reconstructs an `ErrorKind::Other` (`Brain::net_dial` /
    /// `net_open_stream` / `net_stream_send` → `io::Error::other`) and
    /// `pump::peer_outcome` takes the ordinary per-peer arm (drop conn + redial,
    /// round continues, heartbeat advances) — DISTINCT from the brain's OWN
    /// read-deadline `TimedOut` (the A-half poison → supervised-restart signal).
    /// Because the bound fires (10s) well before the brain's 30s
    /// `PUMP_PEER_IO_TIMEOUT`, the broker always REPLIES first. A live op
    /// completes with zero added latency; the bound only bites a non-responsive
    /// peer.
    // [impl->REQ-HAZARD-BROKER-QUIC-DEADLINE]
    fn bounded_block_on<F, T>(&self, fut: F) -> io::Result<T>
    where
        F: std::future::Future<Output = io::Result<T>>,
    {
        let dur = self.quic_op_timeout();
        self.runtime.handle().block_on(async move {
            match tokio::time::timeout(dur, fut).await {
                Ok(inner) => inner,
                Err(_elapsed) => Err(io::Error::other(format!(
                    "broker QUIC op exceeded the {dur:?} bound (peer unresponsive) — \
                     failing as an ordinary per-peer error"
                ))),
            }
        })
    }


    pub fn set_roster_exchange(&self, exchange: RosterExchange) {
        *self.roster.lock().unwrap() = Some(exchange);
    }

    /// The broker-side QUIC-op deadline currently in force (default
    /// [`BROKER_QUIC_OP_TIMEOUT_MS`]).
    fn quic_op_timeout(&self) -> Duration {
        Duration::from_millis(self.quic_op_timeout_ms.load(Ordering::Relaxed))
    }

    /// Override the broker-side QUIC-op deadline (REQ-HAZARD-BROKER-QUIC-DEADLINE).
    /// Production keeps the [`BROKER_QUIC_OP_TIMEOUT_MS`] default; the hermetic
    /// black-hole tests set a short bound so a dead-peer await fails fast instead
    /// of pinning the test for ~10s. Off [`NetConfig`] by design (mirrors
    /// [`Self::set_roster_exchange`]) — the many hermetic test configs need no change.
    // [impl->REQ-HAZARD-BROKER-QUIC-DEADLINE]
    pub fn set_quic_op_timeout(&self, dur: Duration) {
        self.quic_op_timeout_ms
            .store(dur.as_millis() as u64, Ordering::Relaxed);
    }

    /// Run `fut` on the host runtime under the broker-side QUIC-op deadline
    /// (REQ-HAZARD-BROKER-QUIC-DEADLINE) — the B-half of REQ-HAZARD-PUMP-IPC-
    /// DEADLINE. A brain-waiting QUIC await against a black-holed/dead peer must
    /// fail PROMPTLY with an ORDINARY error the broker replies, never the
    /// unbounded await that wedged the single-threaded pump for 2.2h. On elapse
    /// the future is DROPPED — cancelling the in-flight connect / stream op, so
    /// nothing is half-registered — and a non-`TimedOut` `io::Error` is returned.
    /// At the broker boundary that error is stringified into a normal error reply
    /// frame, so the brain reconstructs an `ErrorKind::Other` (`Brain::net_dial` /
    /// `net_open_stream` / `net_stream_send` → `io::Error::other`) and
    /// `pump::peer_outcome` takes the ordinary per-peer arm (drop conn + redial,
    /// round continues, heartbeat advances) — DISTINCT from the brain's OWN
    /// read-deadline `TimedOut` (the A-half poison → supervised-restart signal).
    /// Because the bound fires (10s) well before the brain's 30s
    /// `PUMP_PEER_IO_TIMEOUT`, the broker always REPLIES first. A live op
    /// completes with zero added latency; the bound only bites a non-responsive
    /// peer.
    // [impl->REQ-HAZARD-BROKER-QUIC-DEADLINE]
    fn bounded_block_on<F, T>(&self, fut: F) -> io::Result<T>
    where
        F: std::future::Future<Output = io::Result<T>>,
    {
        let dur = self.quic_op_timeout();
        self.runtime.handle().block_on(async move {
            match tokio::time::timeout(dur, fut).await {
                Ok(inner) => inner,
                Err(_elapsed) => Err(io::Error::other(format!(
                    "broker QUIC op exceeded the {dur:?} bound (peer unresponsive) — \
                     failing as an ordinary per-peer error"
                ))),
            }
        })
    }

+    /// The broker-side QUIC-op deadline currently in force (default

+    /// [`BROKER_QUIC_OP_TIMEOUT_MS`]).

+    fn quic_op_timeout(&self) -> Duration {

+        Duration::from_millis(self.quic_op_timeout_ms.load(Ordering::Relaxed))

+    /// Override the broker-side QUIC-op deadline (REQ-HAZARD-BROKER-QUIC-DEADLINE).

+    /// Production keeps the [`BROKER_QUIC_OP_TIMEOUT_MS`] default; the hermetic

+    /// black-hole tests set a short bound so a dead-peer await fails fast instead

+    /// of pinning the test for ~10s. Off [`NetConfig`] by design (mirrors

+    /// [`Self::set_roster_exchange`]) — the many hermetic test configs need no change.

+    // [impl->REQ-HAZARD-BROKER-QUIC-DEADLINE]

+    pub fn set_quic_op_timeout(&self, dur: Duration) {

+            .store(dur.as_millis() as u64, Ordering::Relaxed);

+    /// Run `fut` on the host runtime under the broker-side QUIC-op deadline

+    /// (REQ-HAZARD-BROKER-QUIC-DEADLINE) — the B-half of REQ-HAZARD-PUMP-IPC-

+    /// DEADLINE. A brain-waiting QUIC await against a black-holed/dead peer must

+    /// fail PROMPTLY with an ORDINARY error the broker replies, never the

+    /// unbounded await that wedged the single-threaded pump for 2.2h. On elapse

+    /// the future is DROPPED — cancelling the in-flight connect / stream op, so

+    /// nothing is half-registered — and a non-`TimedOut` `io::Error` is returned.

+    /// At the broker boundary that error is stringified into a normal error reply

+    /// frame, so the brain reconstructs an `ErrorKind::Other` (`Brain::net_dial` /

+    /// `net_open_stream` / `net_stream_send` → `io::Error::other`) and

+    /// `pump::peer_outcome` takes the ordinary per-peer arm (drop conn + redial,

+    /// round continues, heartbeat advances) — DISTINCT from the brain's OWN

+    /// read-deadline `TimedOut` (the A-half poison → supervised-restart signal).

+    /// Because the bound fires (10s) well before the brain's 30s

+    /// `PUMP_PEER_IO_TIMEOUT`, the broker always REPLIES first. A live op

+    /// completes with zero added latency; the bound only bites a non-responsive

+    fn bounded_block_on<F, T>(&self, fut: F) -> io::Result<T>

+        F: std::future::Future<Output = io::Result<T>>,

+        let dur = self.quic_op_timeout();

+        self.runtime.handle().block_on(async move {

+            match tokio::time::timeout(dur, fut).await {

+                Err(_elapsed) => Err(io::Error::other(format!(

+                    "broker QUIC op exceeded the {dur:?} bound (peer unresponsive) — \

+                     failing as an ordinary per-peer error"

/// Run the joiner leg of the pairing ceremony (M7 D3, REQ-SUBNET-2) on

/// the host's runtime — the broker IPC `pair-join` arm. The ceremony

/// must run HERE because it binds and pins this endpoint's real node


Now wrap the three QUIC await sites. `dial`:


        let self_addr = serde_json::to_value(self.endpoint.addr()).unwrap_or(serde_json::Value::Null);
        self.runtime.handle().block_on(async move {
            let conn = endpoint
                .connect(addr)
                .await
                .map_err(|e| io::Error::other(e.to_string()))?;

        let self_addr = serde_json::to_value(self.endpoint.addr()).unwrap_or(serde_json::Value::Null);
        // Bounded (REQ-HAZARD-BROKER-QUIC-DEADLINE): the `connect` + `prove_membership`
        // awaits below are the exact unbounded sites a dead peer black-holes — wrap
        // them so a non-responsive peer fails as an ordinary error within the bound.
        self.bounded_block_on(async move {
            let conn = endpoint
                .connect(addr)
                .await
                .map_err(|e| io::Error::other(e.to_string()))?;


        let self_addr = serde_json::to_value(self.endpoint.addr()).unwrap_or(serde_json::Value::Null);
        self.runtime.handle().block_on(async move {
            let conn = endpoint
                .connect(addr)
                .await
                .map_err(|e| io::Error::other(e.to_string()))?;

        let self_addr = serde_json::to_value(self.endpoint.addr()).unwrap_or(serde_json::Value::Null);
        // Bounded (REQ-HAZARD-BROKER-QUIC-DEADLINE): the `connect` + `prove_membership`
        // awaits below are the exact unbounded sites a dead peer black-holes — wrap
        // them so a non-responsive peer fails as an ordinary error within the bound.
        self.bounded_block_on(async move {
            let conn = endpoint
                .connect(addr)
                .await
                .map_err(|e| io::Error::other(e.to_string()))?;

// Snapshot the (opt-in) roster exchange + our live address for this dial.

let roster_rx = self.roster.lock().unwrap().clone();

-        self.runtime.handle().block_on(async move {

+        // Bounded (REQ-HAZARD-BROKER-QUIC-DEADLINE): the `connect` + `prove_membership`

+        // awaits below are the exact unbounded sites a dead peer black-holes — wrap

+        // them so a non-responsive peer fails as an ordinary error within the bound.

+        self.bounded_block_on(async move {


`open_stream` Quic branch:


        match backing {
            Backing::Quic(conn) => self.runtime.handle().block_on(async move {
                let (send, recv) = conn
                    .open_bi()
                    .await
                    .map_err(|e| io::Error::other(e.to_string()))?;

        match backing {
            // Bounded (REQ-HAZARD-BROKER-QUIC-DEADLINE): `open_bi` on a black-holed
            // peer can block on stream flow-control credit the dead peer never grants.
            Backing::Quic(conn) => self.bounded_block_on(async move {
                let (send, recv) = conn
                    .open_bi()
                    .await
                    .map_err(|e| io::Error::other(e.to_string()))?;


        match backing {
            Backing::Quic(conn) => self.runtime.handle().block_on(async move {
                let (send, recv) = conn
                    .open_bi()
                    .await
                    .map_err(|e| io::Error::other(e.to_string()))?;

        match backing {
            // Bounded (REQ-HAZARD-BROKER-QUIC-DEADLINE): `open_bi` on a black-holed
            // peer can block on stream flow-control credit the dead peer never grants.
            Backing::Quic(conn) => self.bounded_block_on(async move {
                let (send, recv) = conn
                    .open_bi()
                    .await
                    .map_err(|e| io::Error::other(e.to_string()))?;

-            Backing::Quic(conn) => self.runtime.handle().block_on(async move {

+            // Bounded (REQ-HAZARD-BROKER-QUIC-DEADLINE): `open_bi` on a black-holed

+            // peer can block on stream flow-control credit the dead peer never grants.

+            Backing::Quic(conn) => self.bounded_block_on(async move {


`send_stream` (single block_on covering both transports — bound is harmless for loopback, which never hangs):


        let bytes = bytes.to_vec();
        self.runtime.handle().block_on(async move {
            let mut send = entry.send.lock().await;
            if !bytes.is_empty() {
                send.write_all(&bytes).await?;
            }
            if finish {
                send.finish().await?;
            }
            Ok(())
        })
    }

        let bytes = bytes.to_vec();
        // Bounded (REQ-HAZARD-BROKER-QUIC-DEADLINE): `write_all` / `finish` on a
        // black-holed QUIC stream blocks on flow-control the dead peer never grants.
        // Loopback (in-process) never hangs, so the bound is a no-op for it.
        self.bounded_block_on(async move {
            let mut send = entry.send.lock().await;
            if !bytes.is_empty() {
                send.write_all(&bytes).await?;
            }
            if finish {
                send.finish().await?;
            }
            Ok(())
        })
    }


        let bytes = bytes.to_vec();
        self.runtime.handle().block_on(async move {
            let mut send = entry.send.lock().await;
            if !bytes.is_empty() {
                send.write_all(&bytes).await?;
            }
            if finish {
                send.finish().await?;
            }
            Ok(())
        })
    }

        let bytes = bytes.to_vec();
        // Bounded (REQ-HAZARD-BROKER-QUIC-DEADLINE): `write_all` / `finish` on a
        // black-holed QUIC stream blocks on flow-control the dead peer never grants.
        // Loopback (in-process) never hangs, so the bound is a no-op for it.
        self.bounded_block_on(async move {
            let mut send = entry.send.lock().await;
            if !bytes.is_empty() {
                send.write_all(&bytes).await?;
            }
            if finish {
                send.finish().await?;
            }
            Ok(())
        })
    }

+        // Bounded (REQ-HAZARD-BROKER-QUIC-DEADLINE): `write_all` / `finish` on a

+        // black-holed QUIC stream blocks on flow-control the dead peer never grants.

+        // Loopback (in-process) never hangs, so the bound is a no-op for it.


Add the unit test to nethost's test module.


        let expected: Vec<u8> = frames.concat();
        assert_eq!(got.len(), expected.len(), "no byte lost or duplicated");
        assert_eq!(got, expected, "byte-exact, in order, under backpressure");
    }
}

        let expected: Vec<u8> = frames.concat();
        assert_eq!(got.len(), expected.len(), "no byte lost or duplicated");
        assert_eq!(got, expected, "byte-exact, in order, under backpressure");
    }

    // [unit->REQ-HAZARD-BROKER-QUIC-DEADLINE] the broker-side QUIC-op bound (the
    // B-half of REQ-HAZARD-PUMP-IPC-DEADLINE): `bounded_block_on` returns a PROMPT,
    // ORDINARY (non-`TimedOut`) error on a never-completing op — so the broker can
    // REPLY an error frame instead of leaving the single-threaded pump wedged on a
    // dead peer — while a ready op passes through untouched (the happy path is
    // zero added latency).
    #[test]
    fn bounded_block_on_cuts_a_never_completing_op_with_an_ordinary_error() {
        let host = NetHost::start(hermetic(&Identity::generate())).expect("host start");
        host.set_quic_op_timeout(Duration::from_millis(150));

        // A never-completing await (the black-holed-peer shape) fails at the bound.
        let started = std::time::Instant::now();
        let err = host
            .bounded_block_on(std::future::pending::<io::Result<()>>())
            .expect_err("a never-completing op must error, never block forever");
        let elapsed = started.elapsed();
        assert_ne!(
            err.kind(),
            io::ErrorKind::TimedOut,
            "the bound surfaces an ORDINARY error (peer_outcome's per-peer arm), \
             never the brain's TimedOut poison signal (the A-half restart path)"
        );
        assert!(
            elapsed >= Duration::from_millis(150),
            "it waited at least the bound ({elapsed:?})"
        );
        assert!(
            elapsed < Duration::from_secs(5),
            "and fired promptly, not wildly past it ({elapsed:?})"
        );

        // Happy path untouched: a ready op returns its value with no added latency.
        let v: u64 = host
            .bounded_block_on(async { Ok(7u64) })
            .expect("a ready op passes through");
        assert_eq!(v, 7, "the bound is transparent to a completing op");
    }
}


        let expected: Vec<u8> = frames.concat();
        assert_eq!(got.len(), expected.len(), "no byte lost or duplicated");
        assert_eq!(got, expected, "byte-exact, in order, under backpressure");
    }
}

        let expected: Vec<u8> = frames.concat();
        assert_eq!(got.len(), expected.len(), "no byte lost or duplicated");
        assert_eq!(got, expected, "byte-exact, in order, under backpressure");
    }

    // [unit->REQ-HAZARD-BROKER-QUIC-DEADLINE] the broker-side QUIC-op bound (the
    // B-half of REQ-HAZARD-PUMP-IPC-DEADLINE): `bounded_block_on` returns a PROMPT,
    // ORDINARY (non-`TimedOut`) error on a never-completing op — so the broker can
    // REPLY an error frame instead of leaving the single-threaded pump wedged on a
    // dead peer — while a ready op passes through untouched (the happy path is
    // zero added latency).
    #[test]
    fn bounded_block_on_cuts_a_never_completing_op_with_an_ordinary_error() {
        let host = NetHost::start(hermetic(&Identity::generate())).expect("host start");
        host.set_quic_op_timeout(Duration::from_millis(150));

        // A never-completing await (the black-holed-peer shape) fails at the bound.
        let started = std::time::Instant::now();
        let err = host
            .bounded_block_on(std::future::pending::<io::Result<()>>())
            .expect_err("a never-completing op must error, never block forever");
        let elapsed = started.elapsed();
        assert_ne!(
            err.kind(),
            io::ErrorKind::TimedOut,
            "the bound surfaces an ORDINARY error (peer_outcome's per-peer arm), \
             never the brain's TimedOut poison signal (the A-half restart path)"
        );
        assert!(
            elapsed >= Duration::from_millis(150),
            "it waited at least the bound ({elapsed:?})"
        );
        assert!(
            elapsed < Duration::from_secs(5),
            "and fired promptly, not wildly past it ({elapsed:?})"
        );

        // Happy path untouched: a ready op returns its value with no added latency.
        let v: u64 = host
            .bounded_block_on(async { Ok(7u64) })
            .expect("a ready op passes through");
        assert_eq!(v, 7, "the bound is transparent to a completing op");
    }
}

+    // [unit->REQ-HAZARD-BROKER-QUIC-DEADLINE] the broker-side QUIC-op bound (the

+    // B-half of REQ-HAZARD-PUMP-IPC-DEADLINE): `bounded_block_on` returns a PROMPT,

+    // ORDINARY (non-`TimedOut`) error on a never-completing op — so the broker can

+    // REPLY an error frame instead of leaving the single-threaded pump wedged on a

+    // dead peer — while a ready op passes through untouched (the happy path is

+    fn bounded_block_on_cuts_a_never_completing_op_with_an_ordinary_error() {

+        let host = NetHost::start(hermetic(&Identity::generate())).expect("host start");

+        host.set_quic_op_timeout(Duration::from_millis(150));

+        // A never-completing await (the black-holed-peer shape) fails at the bound.

+        let started = std::time::Instant::now();

+            .bounded_block_on(std::future::pending::<io::Result<()>>())

+            .expect_err("a never-completing op must error, never block forever");

+        let elapsed = started.elapsed();

+            "the bound surfaces an ORDINARY error (peer_outcome's per-peer arm), \

+             never the brain's TimedOut poison signal (the A-half restart path)"

+            elapsed >= Duration::from_millis(150),

+            "it waited at least the bound ({elapsed:?})"

+            elapsed < Duration::from_secs(5),

+            "and fired promptly, not wildly past it ({elapsed:?})"

+        // Happy path untouched: a ready op returns its value with no added latency.

+            .bounded_block_on(async { Ok(7u64) })

+            .expect("a ready op passes through");

+        assert_eq!(v, 7, "the bound is transparent to a completing op");


Build + run the nethost unit tests to validate the impl before writing int tests.


Bash

cargo test -p spt-daemon --lib nethost:: 2>&1 | tail -30

Build + run nethost unit tests


   Compiling spt-proto v0.8.2 (C:\Users\decid\Documents\projects\spt-core\crates\spt-proto)
   Compiling spt-store v0.8.2 (C:\Users\decid\Documents\projects\spt-core\crates\spt-store)
   Compiling spt-msg v0.8.2 (C:\Users\decid\Documents\projects\spt-core\crates\spt-msg)
   Compiling spt-runtime v0.8.2 (C:\Users\decid\Documents\projects\spt-core\crates\spt-runtime)
   Compiling spt-net v0.8.2 (C:\Users\decid\Documents\projects\spt-core\crates\spt-net)
   Compiling spt-term v0.8.2 (C:\Users\decid\Documents\projects\spt-core\crates\spt-term)
   Compiling spt-live v0.8.2 (C:\Users\decid\Documents\projects\spt-core\crates\spt-live)
   Compiling spt-daemon v0.8.2 (C:\Users\decid\Documents\projects\spt-core\crates\spt-daemon)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 31.90s
     Running unittests src\lib.rs (target\debug\deps\spt_daemon-2c4ac76a183cc476.exe)

running 6 tests
test nethost::tests::retentive_log_is_lossless_ordered_while_ordinary_log_overwrites ... ok
test nethost::tests::loopback_conn_is_a_reused_singleton_that_cross_wires_a_stream_pair ... ok
test nethost::tests::host_binds_endpoint_to_node_identity ... ok
test nethost::tests::two_hosts_dial_over_loopback ... ok
test nethost::tests::bounded_block_on_cuts_a_never_completing_op_with_an_ordinary_error ... ok
test nethost::tests::loopback_tunnel_backpressures_without_loss ... ok

test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 308 filtered out; finished in 1.11s


Unit passes, impl compiles. Now the int tests. Check pump internals I need for the regression: `ensure_conn` (does a dial error skip the peer?), `write_heartbeat`/`HEARTBEAT_PERIOD`, and `Broker::journal`/`applied_count`.


cd crates/spt-daemon/src; grep -n "fn ensure_conn\|HEARTBEAT_PERIOD\|fn write_heartbeat\|const TICK\|fn connect\b" pump/mod.rs; echo "=== ensure_conn body ==="; grep -n "fn ensure_conn" -A 30 pump/mod.rs | head -40

Inspect ensure_conn + heartbeat consts


89:const TICK: Duration = Duration::from_millis(200);
94:const HEARTBEAT_PERIOD: Duration = Duration::from_secs(5);
477:        if due(last_hb, false, now, HEARTBEAT_PERIOD) {
668:fn ensure_conn(
732:fn connect(broker_name: &str) -> io::Result<Brain> {
770:fn write_heartbeat(path: &std::path::Path) {
=== ensure_conn body ===
668:fn ensure_conn(
669-    brain: &mut Brain,
670-    conns: &mut HashMap<String, u64>,
671-    resolver: &PeerResolver,
672-    peer_hex: &str,
673-    peer_addrs: &std::path::Path,
674-) -> Option<u64> {
675-    if let Some(id) = conns.get(peer_hex) {
676-        return Some(*id);
677-    }
678-    let conn_id = dial_seeded(resolver, peer_hex, peer_addrs, |addr| {
679-        brain
680-            .net_dial(addr, None)
681-            .map(|d| (d.conn_id, d.remote_addr))
682-    })?;
683-    conns.insert(peer_hex.to_string(), conn_id);
684-    Some(conn_id)
685-}
686-
687-/// The seeded dial plan (M8 decisions 14/20): try the durable last-known
688-/// address, then the injected resolver — first success wins, persisting the
689-/// dial reply's OBSERVED address back into the seed file. `dial` returns the
690-/// broker-minted conn id + the observed remote addr (`null` = nothing worth
691-/// seeding). Factored from [`ensure_conn`] so the resolution ORDER is a unit
692-/// fact, not a prose promise.
693-// [impl->REQ-CONV-1]
694-fn dial_seeded(
695-    resolver: &PeerResolver,
696-    peer_hex: &str,
697-    peer_addrs: &std::path::Path,
698-    mut dial: impl FnMut(serde_json::Value) -> io::Result<(u64, serde_json::Value)>,


`ensure_conn` → dial error → `None` → skip peer, round continues. `net_dial(addr, None)` (non-journaled) in the pump, so the journal/exactly-once facet is covered by the netbroker.rs test (with `Some(OP)`). `HEARTBEAT_PERIOD = 5s`. Read the heartbeat format.


770	fn write_heartbeat(path: &std::path::Path) {
771	    if let Some(parent) = path.parent() {
772	        let _ = std::fs::create_dir_all(parent);
773	    }
774	    let _ = spt_store::atomic::atomic_write_string(path, &now_ms().to_string());
775	}
776
777	/// The supervisor's next backoff after a death: a healthy run (survived
778	/// [`SUPERVISE_HEALTHY_RUN`]) resets to the floor `base`; a fast death
779	/// doubles, capped at [`SUPERVISE_BACKOFF_CAP`] (M8 decision 23 — a
780	/// persistently-crashing pump stays visible and bounded, never silent and
781	/// never a hot crash loop). Pure for the unit table.
782	// [impl->REQ-DAEMON-5]
783	fn next_backoff(prev: Duration, ran_for: Duration, base: Duration) -> Duration {
784	    if ran_for >= SUPERVISE_HEALTHY_RUN {
785	        base
786	    } else {
787	        prev.saturating_mul(2).min(SUPERVISE_BACKOFF_CAP)
788	    }
789	}
790
791	/// Supervise repeated runs of a pump body until `stop` is raised (M8
792	/// decision 23, REQ-DAEMON-5): a panic is CAUGHT and logged loud
793	/// (`PEER_PUMP_PANIC`), an error return logged (`PEER_PUMP_FAIL`), and the
794	/// body restarts after a capped, doubling backoff — a 5.9-class pump death
795	/// self-heals visibly instead of silently halving the daemon for 24 h.
796	/// `base` is the backoff floor (production: [`SUPERVISE_BACKOFF_BASE`];
797	/// the unit harness shrinks it). Backoff sleeps in [`TICK`] slices so a
798	/// stop request lands promptly. Generic over the body for the unit
799	/// harness; production binds [`run_peer_pump`].
800	// [impl->REQ-DAEMON-5]
801	pub fn supervise_pump(stop: &AtomicBool, base: Duration, mut body: impl FnMut() -> io::Result<()>) {
802	    let mut backoff = base;
803	    while !stop.load(Ordering::Relaxed) {
804	        let started = Instant::now();

fn write_heartbeat(path: &std::path::Path) {
    if let Some(parent) = path.parent() {
        let _ = std::fs::create_dir_all(parent);
    }
    let _ = spt_store::atomic::atomic_write_string(path, &now_ms().to_string());
}

/// The supervisor's next backoff after a death: a healthy run (survived
/// [`SUPERVISE_HEALTHY_RUN`]) resets to the floor `base`; a fast death
/// doubles, capped at [`SUPERVISE_BACKOFF_CAP`] (M8 decision 23 — a
/// persistently-crashing pump stays visible and bounded, never silent and
/// never a hot crash loop). Pure for the unit table.
// [impl->REQ-DAEMON-5]
fn next_backoff(prev: Duration, ran_for: Duration, base: Duration) -> Duration {
    if ran_for >= SUPERVISE_HEALTHY_RUN {
        base
    } else {
        prev.saturating_mul(2).min(SUPERVISE_BACKOFF_CAP)
    }
}

/// Supervise repeated runs of a pump body until `stop` is raised (M8
/// decision 23, REQ-DAEMON-5): a panic is CAUGHT and logged loud
/// (`PEER_PUMP_PANIC`), an error return logged (`PEER_PUMP_FAIL`), and the
/// body restarts after a capped, doubling backoff — a 5.9-class pump death
/// self-heals visibly instead of silently halving the daemon for 24 h.
/// `base` is the backoff floor (production: [`SUPERVISE_BACKOFF_BASE`];
/// the unit harness shrinks it). Backoff sleeps in [`TICK`] slices so a
/// stop request lands promptly. Generic over the body for the unit
/// harness; production binds [`run_peer_pump`].
// [impl->REQ-DAEMON-5]
pub fn supervise_pump(stop: &AtomicBool, base: Duration, mut body: impl FnMut() -> io::Result<()>) {
    let mut backoff = base;
    while !stop.load(Ordering::Relaxed) {
        let started = Instant::now();


The faithful black hole is a membership-mismatch (A requires the proof, B never answers it) — the documented mixed-pair deadlock that hangs A's dial in `prove_membership`. That's the actual DEAD-peer dial shape. Add the `proving` helper + the broker-level int to netbroker.rs.


Edit

/// Bind a broker (serving on a background thread) that owns a hermetic endpoint
/// for `identity`, with its effect journal in `dir`.
fn net_broker(name: &str, identity: Identity, dir: &std::path::Path) -> Arc<Broker> {

/// A hermetic config that REQUIRES the mutual seed-proof (`membership: Some`):
/// a dial then runs `prove_membership`, which a `membership: None` peer never
/// answers — the documented mixed-pair deadlock (the dialer opens a proof
/// control stream and awaits a reply that never comes). The credential content
/// is irrelevant; its PRESENCE is what makes the dialer open the stream and
/// block, giving the deterministic black hole the broker-side bound must cut.
fn proving(identity: Identity) -> NetConfig {
    let mut cfg = hermetic(identity);
    cfg.membership = Some(std::sync::Arc::new(|| {
        vec![spt_daemon::seedproofx::SubnetCred {
            name: "blackhole".to_string(),
            seed: vec![7u8; 32],
            epoch: 1,
            prev: None,
        }]
    }));
    cfg
}

/// Bind a broker (serving on a background thread) that owns a hermetic endpoint
/// for `identity`, with its effect journal in `dir`.
fn net_broker(name: &str, identity: Identity, dir: &std::path::Path) -> Arc<Broker> {