# PUMP-TRUTH W2 — DESIGN (todlando, from doyle's dispatch ruling @d806932)

`REQ-PUMP-PEER-ISOLATION`: concurrent non-blocking dials + per-peer fault isolation on the D4c presence-event seam. **Design-only** (no build; W1 gates first, box is deployah's for musl CI). For doyle's design-check before mint+build.

## Premise check (CONTEXT / ADRs)
- **D4c presence seam is the sanctioned substrate.** `PresenceLog` (nethost.rs:418, ring `DEFAULT_STREAM_RING_CHUNKS`, cursor-resumable, gapless exactly-once, survives brain restart — nethost.rs:279) already carries `PRESENCE_CONNECTED` on every `register_conn` (nethost.rs:458-461). `NetPresenceEvent.kind` is a **string, explicitly so a newer broker can add kinds an older brain tolerates** (msg.rs:904) — adding a dial-outcome kind is the designed forward-compat path, not a wire break.
- **ADR-0018 §V4 stagger** (`mark_ran` = ATTEMPTED, post-round instant) and **REQ-HAZARD-PUMP-IPC-DEADLINE** (bounded read → no infinite wedge) are preserved, the latter **re-pointed** to broker-conn liveness (below), not removed.
- **ADR-0034** journal namespacing: dial-submit keeps the existing `op_id`/`minter` exactly-once dial dedup (NetDialReq already carries them).
- Additive: the blocking `net_dial` request-reply STAYS for its other callers (shellwake, pairing); W2 ADDS the submit+event path the pump uses.

## Wire changes (`crates/spt-daemon/src/msg.rs`)

1. **New presence kind** `PRESENCE_DIAL_FAILED = "dial-failed"` (alongside `connected`/`disconnected`). Ride the existing `NetPresenceEvent`; add two optional, additive fields (absent on the existing kinds → old brains unaffected):
   - `reason: Option<String>` — the ordinary per-peer failure text (W1's bounded error, or "no route"), for the log + per-peer backoff decision.
   - `remote_id_hex` is ALREADY on `NetPresenceEvent` → **the per-peer correlation key** (G1). A failed dial has no `conn_id` → set `conn_id: 0` (documented sentinel; consumers key on `kind` + `remote_id_hex`, never `conn_id` for a fail).
   - (Correlation is by `remote_id_hex`; the pump submits per `peer_hex` and matches outcomes by it. A `submit_id` is NOT needed — a peer has at most one in-flight dial per round, enforced by the pump's per-peer state.)
2. **New non-blocking request** `KIND_NET_DIAL_SUBMIT = "net-dial-submit"`, payload = the existing `NetDialReq` (addr + op_id + minter — unchanged shape). The broker acks immediately (a bare ok/received), dials concurrently, and the OUTCOME rides the presence stream (CONNECTED on success — already emitted by `register_conn`; DIAL_FAILED on failure — new). No blocking reply.

## Broker changes (`broker.rs` + `nethost.rs`)

3. **`NetHost::submit_dial(addr, op_id, minter)`** (new, non-blocking) — spawns the dial on the host runtime (`self.runtime.spawn`), returns immediately. The spawned task:
   - acquires a **bounded semaphore permit** (G4: a NetHost-shared `tokio::sync::Semaphore`, cap `PUMP_DIAL_CONCURRENCY` = 16–32) — a large-roster guard, natural since the broker is already async iroh. **CAVEAT (doyle Q2, load-bearing):** the existing blocking `dial()` MUST acquire the SAME permit too, or the cap only bounds `submit_dial` while `net_dial` piles on unbounded — both dial paths draw from the one NetHost semaphore. Large-roster > cap: dials queued behind a permit resolve past the round deadline → left for the next round (per-peer `next_due`), never a round failure.
   - runs the SAME bounded `connect + prove_membership` as `dial()` (W1's per-dial bound applies unchanged) under the journal (`op_id`/`minter` dedup).
   - on success → `register_conn` (emits `PRESENCE_CONNECTED` as today).
   - on failure → append `PRESENCE_DIAL_FAILED { remote_id_hex, reason }` to the same `PresenceLog` (so the outcome rides the D4c cursor — a mid-round broker restart replays it; no peer is silently never-rescheduled).
4. **`dispatch_net_dial_submit`** (broker.rs, beside `dispatch_net_dial` :3658) — decodes `NetDialReq`, calls `submit_dial`, replies the immediate ack. The blocking `dispatch_net_dial` is untouched.

## Pump changes (`pump/mod.rs` — `run_peer_pump` restructure)

5. **Subscribe once** to the presence stream (`NetPresenceSubscribeReq { from_seq }`) at pump start, resuming the cursor across supervised restarts (exactly-once).
6. **Per-peer state** — `HashMap<peer_hex, PeerSched { next_due: Instant, backoff: Duration }>` (G5). A peer is dial-eligible this round iff `now >= next_due`. On `DIAL_FAILED` → `backoff = (backoff*2).min(PEER_BACKOFF_CAP≈300s)`, `next_due = now + backoff`. On `CONNECTED` (success) → reset (`backoff = base`, remove/zero). Live peers stay hot; a persistently-dead peer backs off, never re-dialed every tick.
7. **Round = submit-all → drain-until-deadline → mark_ran** (G3, wedge-proof):
   - compute `due_flags` (cadence, unchanged) + `pre_rounds` (unchanged).
   - `fan_targets` → for each peer NOT in backoff and NOT already-connected → **submit** a non-blocking dial (fire-and-forget). Concurrent on the broker.
   - **drain** the presence stream until a **round deadline** `ROUND_DRAIN_TIMEOUT` = a **derived const** `W1_final_bound + ~2s margin` (doyle Q1: NOT a config knob — reference W1's bound constant directly, **pin the value when W1 lands its final bound**; the margin covers event IPC transit + drain scheduling; concurrent dials all resolve within ~one bound, not N×bound). For each event:
     - `CONNECTED{remote,conn_id}` → record conn; run that peer's due-worker `peer_step`s (the registry advertise / notif / sync / update legs) on its conn — **the live peer advertises presence the round it connects**, independent of offline peers. **If a `peer_step` errors mid-round** (a CONNECTED peer whose stream write fails — doyle Q3): drop that conn + apply the per-peer backoff, the SAME path as `DIAL_FAILED`, never round-wide (bubbling it round-wide would re-introduce the whole-round poison in a new spot). No transient-vs-dead distinction — a recovered peer's next successful connect resets its backoff.
     - `DIAL_FAILED{remote,reason}` → per-peer backoff (6); log `PUMP_PEER_FAIL:{remote}:{reason}` (the ordinary per-peer signal, now event-driven).
     - `DISCONNECTED{conn}` → drop the cached conn (redial-eligible next round).
   - at the deadline → `mark_ran` (post-round instant, §V4 stagger). Peers unresolved at close = left for a future round (per-peer `next_due`), **never a round failure**.
8. **DELETE the whole-round poison** — `peer_outcome(...)?` (mod.rs:531) and its `TimedOut → Err` bubble go away. A dead peer is a `DIAL_FAILED` event, never a bubbled `Err`, never aborts the round, never restarts the pump.

## Re-pointed REQ-HAZARD-PUMP-IPC-DEADLINE (doyle's explicit check: don't lose dead-broker detection)

The pump's brain carrier keeps a **short liveness deadline on the DRAIN read** — but now it means "is the BROKER answering the event stream AT ALL," not "did peer X reply." Concretely:
- a healthy round always produces stream activity (at minimum the pump's own submit acks + the round's outcome events) within the liveness window;
- if the drain read deadlines with the broker producing **nothing** (broker wedged / carrier dead) → `run_peer_pump` returns `Err` → `supervise_pump` restarts (the ORIGINAL 2.2h-wedge defense, intact);
- a dead PEER can no longer trip this — its `DIAL_FAILED` arrives within W1's bound as ordinary stream traffic. **Decouple achieved without losing the dead-broker restart.**
- The int test asserts BOTH directions: a dead peer never restarts; a dead broker conn STILL does.

## Exactly-once / gapless (G1 rationale, preserved)
Dial-outcomes live in the SAME `PresenceLog` ring+cursor as connected/disconnected. A mid-round broker restart → the pump resumes from `from_seq` and replays any un-consumed outcome → no peer silently never-rescheduled, no conn double-used. One cursor, one ordering.

## Tests (gate)
- **int (load-bearing, `tests/`):** mixed roster **1 LIVE + N OFFLINE** — the live peer CONNECTED + this node advertises to it in the SAME round the N offline peers DIAL_FAILED; heartbeat advances every round; **no `PEER_PUMP_RESTART` from a dead peer**; the live peer is never head-of-line-blocked by a dead one. Plus: a dead BROKER conn STILL trips `supervise_pump` (re-pointed deadline intact). `[int->REQ-PUMP-PEER-ISOLATION]`.
- **unit:** per-peer doubling backoff (double on fail, reset on success, cap); correlation-by-`remote_id_hex` (outcomes matched out of submit order); round closes on the deadline with a never-resolving peer (wedge-proof); `DIAL_FAILED` never bubbles a whole-round Err. `[unit->REQ-PUMP-PEER-ISOLATION]`.
- **impl tags** on: the new msg kinds/fields, `submit_dial` + semaphore, `dispatch_net_dial_submit`, the restructured round loop, the per-peer sched.

## Files
`msg.rs` (kinds+fields+submit req), `nethost.rs` (`submit_dial`+semaphore+DIAL_FAILED append), `broker.rs` (`dispatch_net_dial_submit`), `brain.rs` (submit helper + the presence-drain already exists), `pump/mod.rs` (round restructure + per-peer sched), tests. Activate `REQ-PUMP-PEER-ISOLATION [impl,unit,int]` at build-start.

## Q1–Q3 — RESOLVED (doyle design-check addendum, PUMP-W2-DISPATCH-RULING.md @ca2038b)
- **Q1 (round deadline):** RESOLVED — a **derived const** `W1_final_bound + ~2s`, NOT a config knob. Reference W1's bound constant; pin at W1 land (genuine W1-before-W2 dependency — don't hardcode 10s now). Folded into round loop above.
- **Q2 (semaphore home):** RESOLVED — **NetHost-shared** (guards the broker's iroh conn resource; protects all dial callers). CAVEAT: the blocking `dial()` draws the SAME permit. Folded into `submit_dial` above.
- **Q3 (peer_step mid-round fail):** RESOLVED — per-peer drop+backoff, SAME path as `DIAL_FAILED`, never round-wide (my proposal, endorsed). Folded into the CONNECTED handler above.
- **Keep-confirmations (doyle):** (a) `DISCONNECTED` stays NO-backoff — a previously-reachable peer that drops is redial-eligible immediately next round; only `DIAL_FAILED` applies backoff. (b) large-roster > cap → queued dials resolve past the deadline → next round, never a round failure.

**Design is build-ready.** Mint `REQ-PUMP-PEER-ISOLATION [impl,unit,int]` + build in a `.worktrees/` worktree POST-W1-gate.