# Broker/Brain Process-Isolation Restoration — Milestone Build Plan

> **Status: RATIFIED (2026-06-09).** Design (`docs/BROKER-BRAIN-SPLIT-RESTORATION.md`,
> verified-with-amendments by `doyle`) and decision (`docs/adr/0018-...`) are
> operator-ratified; sequencing ratified as the **next** milestone, before
> `spt-claude-code`. REQs minted inactive (TRACEABILITY rule 3): this plan is
> the build, and schedules their activation per task (rule 5: activate at the
> commit that delivers the evidence). This is the **last** release that needs
> a manual fleet daemon bounce.

Goal: restore ADR-0004's intended **two-process** model so a routine
brain-only update hands off to the new executable seamlessly — no endpoint
or harness child dropped (REQ-UPD-3), loop timings continuous across the
swap, failures fail *safe*, uniform on Windows + Linux. The production daemon
today runs the broker as a background **thread** inside the single `spt
daemon` process (`daemon.rs:165-170`), so `apply` swaps the binary on disk
but never restarts the running code — the no-endpoint-drop self-update pillar
is silently unrealized. Restore it as a real process boundary.

This changes daemon **internals only** — the M8-frozen CLI/api surface is
untouched, so the adapter window stays open and `spt-claude-code` lands on
the final topology.

## Why now (sequencing — ratified)

1. **Queue open** — mesh shipped v0.3.0, M8 acceptance COMPLETE.
2. **Self-bootstrapping (decisive)** — until this ships, *every* release needs
   a manual fleet-wide daemon bounce (paid 3× on v0.3.2; `enlyzeam` ran 0.3.0
   with the 0.3.2 fix on disk for ~a day, still reproducing the `\r`-corruption).
   The restoration release is the last needing a manual bounce.
3. **Adapter window safe** — internals only; better the adapter lands on the
   final topology than atop a daemon under later open-heart surgery while it
   hosts the user's daily driver.

## What "done" looks like (goal-backward)

- `spt daemon run` is the **broker** process: binds seed-control + broker
  socket + NetHost + digest hub; holds PTY masters / harness children /
  `EffectJournal`; spawns the **brain** as a child and supervises it.
- A hidden `spt daemon brain` entry is the brain: connects to the broker over
  socket IPC, runs the logic loops, rehydrates from disk, emits `ready`.
- `spt update apply` swaps the binary, signals the brain to snapshot +
  self-exit, the broker respawns from the executable path (now the new
  binary), the new brain re-attaches **all** sessions in resume mode — and a
  live PTY child's pid is unchanged and its QUIC conn survives, at the
  **process** level. New code runs immediately, no logon/manual bounce.
- A new brain that fails to reach `ready` within the bounded window
  auto-rolls-back to `spt.exe.old-N`, quarantines the bad version, and fires a
  loud notif — endpoints alive throughout.
- `ensure_running` / `is_running` / `daemon stop` contracts unchanged (target
  the broker's seed channel); de-elevation guard applies at the broker entry.

---

## Task decomposition (dependency-ordered)

Each task maps to ADR-0018 decisions (Q2–Q8, amendments V1–V6) and lands as
its own atomic commit(s) with evidence tagged in-commit. Build order is the
dependency order: the process split is foundational; rollback + the proof
come last.

### D1 — Process split skeleton (foundation) · Q8, §5

The structural cut, proven minimal before any logic moves. **D1 is a *true*
minimal skeleton** (doyle vet 2026-06-09, finding 1): the brain child runs
ONLY the path that already speaks socket IPC; every direct-`Arc<Broker>`
consumer (pump `daemon.rs:280`, dispatcher, digest *parse*, net consumers,
psyche loops — design §2.2) **stays in the broker process** until D2 gives it
a verb. Those loops hold direct `Arc` calls, so they *cannot* run
cross-process before D2's verb conversion — moving them in D1 would wall the
implementing agent.

- Add the hidden **`spt daemon brain`** subcommand (brain process entry).
  `spt daemon run` stays the user-facing entry but becomes the **broker**
  process: it binds the seed-control channel + broker socket + NetHost +
  digest hub (as today), then **`Command::spawn`s the brain child**.
- **What runs where at D1-close (the falsifiability anchor):**
  - **Brain child:** ONLY `Brain::cold_start` (connect + IPC handshake over
    the existing socket) + session attach — the already-socket-speaking path.
    Emits a **`ready`** signal once attached. (`connect_retry`,
    `applyhost.rs`, is the reused brain-side connect primitive.)
  - **Broker process (temporarily):** pump, dispatcher, psyche loops, digest
    parse, net consumers — they still hold their `Arc<Broker>` and keep
    serving in-process exactly as today. D2 migrates them out one at a time.
- Broker ↔ brain talk over the **existing local socket IPC** (the versioned
  contract already carrying `SPAWN`/`INPUT`/`NET_*`/`SESSIONS`). No `exec`,
  no FD-passing → no Windows/Unix divergence (Q8).
- De-elevation guard (`daemon.rs:60-103`) stays at the **broker** entry; the
  brain child inherits the unelevated identity.
- **Acceptance (this IS de-risk spike #1 re-proven in production topology):**
  `spt daemon run` brings up a broker process + a distinct brain-child pid;
  killing the brain leaves the broker (and any PTY child) alive and the broker
  respawns it. "No behavior change visible at the CLI" is **falsifiable**
  precisely because D1 states which loops run where: the broker-resident loops
  still serve while the brain is the thin attach layer.

### D2 — IPC verb surface + N-1 compat scaffolding · Q5, V6

With two processes there is no shared `Arc<Broker>`; every brain→broker
direct call becomes a versioned IPC verb on the same contract. **D2 also
*relocates* the loops D1 left in the broker process** (pump/dispatch/psyche/
digest-parse/net-consumers) into the brain child — one per commit, as its
verb lands.

- **Migration ledger (loop → verb → moves-at-commit).** Each commit moves
  exactly ONE consumer: define its versioned verb, reroute the direct call
  over the socket, move the loop into the brain child, prove the N-1 window.
  Keep the ledger current in-plan so "which loop runs where" is answerable at
  every mid-migration commit. **D2-close invariant:** every logic loop runs in
  the brain; the broker holds only continuity resources + the verb server.
- **Audit** every current in-process `Arc<Broker>` consumer; convert each
  direct call to a versioned verb. Per §Q5:
  - **Net bring-up + REQ-DAEMON-9 self-heal retry → broker** — *near-free*:
    NetHost is already broker-owned (`broker.rs:175` `OnceLock`), and net
    bring-up already lives in the entry that *becomes* the broker entry.
  - **Digest hub → broker** (live projections + subscriber sockets); **the
    parse that feeds it → brain** (brain pushes parsed digests over IPC).
  - **Seed-lock + liveness → broker** (the Q2 anchor; see D3).
  - Every other brain→broker call → a versioned verb, defaulted/additive per
    KH-2.3 (handoff-argv-compat).
- **[V6] N-1 compat is milestone work.** Steady state after every routine
  update is **new-brain × old-broker** (broker almost never updates). The
  `classify` pre-swap handshake (`brain_ipc_version ≥ broker.min_compatible`)
  gates the forward direction, but each verb conversion must hold the N-1
  window. Build the **CI-real compat test** here: an **old-broker binary ×
  new-brain binary** exercise across the whole verb surface (else KH-2.3
  returns the first time a verb signature changes). Scaffold it in D2; it
  becomes a gate in D7.
- **Exception (Q5):** shellwake `wake_command` watcher **children stay
  brain-owned**, re-reconciled from disk on brain start (the boot sweep at
  `daemon.rs:206-217` already does orphan cleanup). Accepted: a rare brain
  update may briefly miss a wake for an offline shell.

### D3 — Supervision anchor + update trigger · Q2, Q3, V2

The broker becomes the always-up per-machine anchor and the brain's supervisor.

- **Seed-control lock + liveness move to the broker** (Q2): the
  single-daemon-per-`SPT_HOME` invariant and the `ensure_running`/`is_running`
  ping target live in the never-restarting layer, so a brain restart can't let
  a second daemon win the bind or make `is_running` flap. `ensure_running` /
  `is_running` / `daemon stop` external contracts are **unchanged** — they
  target the broker's seed channel (which is where they already point).
- **Broker supervises + respawns the brain** as its child (the crash-recovery
  path it must already provide).
- **Update trigger (Q3):** `apply` verifies + swaps the binary on disk, then
  signals the brain to **snapshot + self-exit**. The broker observes the exit
  and **respawns from the executable path (now the new binary)**; the new
  brain re-attaches. This reuses the existing `apply_brain_only` snapshot→drop→
  re-attach primitive — an update is a *planned* crash on the path the broker
  already recovers from. (`update.rs:233-234`'s "exec the new binary's brain"
  finally becomes real, via respawn rather than exec.)
- **[V2] Generation custody → broker** (Q2): retiring the `BrainState`
  message (D4) would orphan the KH-2.4 generation counter. The broker owns it,
  increments on **every** brain spawn (planned or crash), and hands
  `{generation, start-reason}` to the brain at spawn time via a **versioned
  argv/hello field** (KH-2.3 forward-compat, defaulted). Because the broker
  observes every respawn this is strictly more reliable than brain→brain.
- The **same spawn-time channel carries Q4's update-vs-crash discriminator**
  (one channel, both payloads): the broker *initiated* a planned cycle but
  *observed* an unexpected crash.

### D4 — Multi-session resume; retire the `BrainState` message · Q6

Under the self-exit trigger the outgoing brain is gone before the new one
starts, so continuity must come from the persistent side, not a brain→brain
frame.

- The **broker becomes cursor-of-record** per session-subscriber (it already
  holds the per-session output ring, `broker.rs:73-143`): track each
  subscriber's last-**delivered** cursor.
- On (re)start the new brain queries the broker for **all** hosted sessions
  and re-attaches each in **resume** mode — fixing today's two gaps:
  `applyhost.rs:239` re-attaches only `sessions.first()` (one session) with
  `from_seq=0` (re-replays the whole ring → duplicate output).
- **Output is at-least-once** (broker resumes from last-*sent*); **input/
  effects stay exactly-once** via the already-broker-owned `EffectJournal`.
  Matches the SPIKE-05 terminal-stream contract (a resize repaint already
  reorders/duplicates the stream; exactly-once output would need a per-chunk
  ack for a guarantee the stream doesn't make).
- The explicit **`BrainState` handoff *message* retires** for production: the
  new brain cold-starts and reconstructs continuity by querying the broker.
  (`brain.rs` `BrainState`/`handoff` stay available for tests; the production
  path stops passing the frame.)

### D5 — Durable absolute-deadline loop timing · Q4, V3, V4

Timing that must survive a swap lives as durable absolute-deadline state on
disk, rehydrated on every brain start — not in the (planned-only) handoff
snapshot, which a crash carries nothing of.

- **Phase-significant resident loops only [V4].** The **pulse loop**
  (`lifecycle.rs:486` `sleep_interruptible(pulse_period)`, phase-relative →
  resets on restart) converts to a disk-anchored deadline. The **idempotent
  pump cadences need NO conversion** — they already "stagger from everything
  due now" (`peerloop.rs:304`), catch-up/idempotent, restart-safe; converting
  them would reintroduce the per-loop writes Q4 minimized.
- **Mechanism:** persist `(anchor, interval)` **once** on a fresh/crash start;
  derive `next_fire = anchor + interval × ⌈max(0, now − anchor) / interval⌉` —
  **no per-fire writes**. An **update restart re-reads `(anchor, interval)`
  and keeps deriving** (phase preserved, lands mid-grid). A **crash restart is
  a fresh start** (rewrite `anchor = now`; phase reset acceptable). The brain
  distinguishes update-vs-crash via the D3 spawn-time discriminator.
- **One-shot deadlines (alarms): rule fixed here, machinery deferred [V3].**
  Persist the absolute `target-time` at creation; every start (update *or*
  crash) reads it and fires-if-due; **never reset on crash** ("remind me at
  3pm" is a commitment). But the daemon has **no one-shot consumer today**
  (alarms are legacy-listener-only, §7) — building it now ships untested dead
  code. The *rule* is specified this milestone; the *machinery* is built with
  the alarm port (activate-don't-pre-fail). Tracked in DEFERRED.md
  ("Durable in-daemon alarm scheduler" row).

### D6 — Failure atomicity: readiness-gated auto-rollback · Q7, V1

A new brain that bricks *logic* (panics on boot) must self-heal without a
human — endpoints stay up (the broker holds them).

- **Bounded-retry → auto-rollback.** If the new brain fails to reach `ready`
  (re-attached all sessions + resumed loops) within a bounded number of boots
  / a healthy-run window, the broker **rolls back to the last-known-good
  binary** (`spt.exe.old-N`), **quarantines** the bad version (no auto
  re-apply/re-fetch), and surfaces a loud consent-style notification. The
  broker holds **both** last-known-good + candidate paths and chooses which
  to spawn (no file manipulation at failure time). **Reuse** the existing
  readiness+backoff machinery: `peerloop.rs:82-91`
  `SUPERVISE_BACKOFF_BASE/CAP` + `SUPERVISE_HEALTHY_RUN` (60 s healthy resets
  backoff), `supervise_pump` wired at `peerloop.rs:805` (M8-D4).
- **Two-phase applied record** (fixes a real existing bug): `applyhost.rs:176`
  writes `record_applied(version)` + `last-outcome=applied` **before** the
  handoff — the optimistic `applied.json={version:6}` observed on `enlyzeam`.
  Make it: `applied-pending` at swap → promoted to `applied` on the `ready`
  signal → corrected to `rolled-back(quarantine=N, running=N-1)` on failure.
- **[V1] Rollback-state-compat invariant.** Auto-rollback spawns the **old**
  binary against durable state the **new** brain may already have written.
  Safe today (2026-06-09 audit: zero state-migration code), but the first
  durable-state schema migration would silently break rollback. Assert the
  invariant **now, while free**: a brain must not irreversibly migrate durable
  state before ready-promotion — every pre-ready write stays N-1-readable.
  This task lands the invariant as an explicit, tested guard/marker (and a
  doc note in KNOWN-HAZARDS 6.8) so a future migration trips a known wire.

### D7 — Proof + REQ re-pointing + fleet verify · V5, V6, §9

The milestone's conformance close-out.

- **Productionize SPIKE-01/03 as an `int` E2E [V5]:** a PTY child + a live
  QUIC conn survive a brain-**process** restart **onto a swapped binary** —
  pid unchanged, output stream gapless, conn intact. This is the test that
  proves *process-level* survival (the spikes proved the shape with two
  binaries; this proves it in production topology).
- **Re-point the regression-masked evidence.** REQ-DAEMON-2 and REQ-UPD-3's
  current `int` tags prove only the *in-process* handoff shape
  (`applyhost.rs:39/77`, `update.rs:246`, `brain_swap.rs`, the M3b-B9 E2E) —
  mis-evidenced today. Move them to the D7 process-level E2E.
- **N-1 compat test as a CI gate [V6]:** the old-broker × new-brain
  verb-surface exercise scaffolded in D2 becomes a green gate.
- **Activate the minted hazards:** set real `required_stages` on
  `REQ-HAZARD-BROKER-PROCESS-ISOLATION` (doc=ADR-0018+design; impl/unit across
  D1–D6; int=the D7 E2E + N-1 test) and `REQ-HAZARD-ROLLBACK-STATE-COMPAT`
  (doc=ADR-0018+KH 6.8; impl/unit=D6 invariant guard).
- **Fleet verify** the real seamless update on the 3-node rig
  (kitsubito/hfenduleam/enlyzeam): publish a restoration+1 test release →
  `apply` on each node → **the running pid changes / new code goes live with
  no manual bounce**, every hosted endpoint survives. This is the acceptance
  the v0.3.2 roll could not get for free.

---

## Traceability — REQ activation schedule

| REQ | Today | This milestone | Activated at |
|-----|-------|----------------|--------------|
| `REQ-HAZARD-BROKER-PROCESS-ISOLATION` | minted, `required_stages=[]` | doc → impl → unit → int | doc now (ADR-0018+design exist); impl/unit across D1–D6; **int at D7** (process-level E2E + N-1 test) |
| `REQ-HAZARD-ROLLBACK-STATE-COMPAT` | minted, `required_stages=[]` | doc → impl → unit | doc now (ADR-0018+KH 6.8); **impl/unit at D6** (invariant guard) |
| `REQ-UPD-3` (no endpoint drop, brain-only) | `int` proves in-process shape (mis-evidenced) | re-point `int` to process-level survival | **D7** |
| `REQ-DAEMON-2` (broker/brain split for seamless update) | `int` proves in-process shape | re-point `int` to process-level survival | **D7** |

Rule 5 (activate-don't-pre-fail): toml `required_stages` are set at the
commit that delivers each stage's evidence, **not** now — `traceable-reqs
check` stays green at every commit. Re-pointing REQ-DAEMON-2/UPD-3 happens in
the same D7 commit that lands the replacement E2E (never leave them evidenced
by the old in-process test once the new test exists).

## Per-task gates (every commit)

`cargo build` · `cargo test` · `cargo clippy` · `cargo build
--no-default-features` · `traceable-reqs check` (EXIT=0) · `xtask check` (doc
drift). Push to a dev-freeform branch → **CI both runners** before any
release tag (the v0.3.0 lesson: CI catches regressions a local box can't —
single-home non-elevated dev box is blind to service-detection / socket-path
issues). The `[twohost]` rig rungs gate D7.

## Risks / watch-items

- **The hardest invariant** (de-risk spike #1): a PTY child + open socket
  survive a brain-process restart. The spikes proved it; D1 must re-prove it
  in production topology before logic moves (D3+). If D1's acceptance fails,
  stop — the whole design rests on it.
- **N-1 verb drift (KH-2.3).** Every verb added in D2 must default/additive;
  the D7 N-1 gate is the backstop. A breaking verb signature change is a
  broker-breaking update class — out of scope, must be refused, not silently
  shipped.
- **Test broker socket-bind flake** (DEFERRED, 2026-06-09): the AF_UNIX
  `served_broker` bind flakes on kitsubito. D1/D7 add real process-spawn
  tests on that runner — fix the bind determinism (unlink-before-bind /
  abstract namespace / per-test TempDir) opportunistically here rather than
  fighting flakes through the milestone.
- **Generation counter (KH-2.4):** custody moves to the broker (V2). Verify no
  consumer reads `gen_start_ms` as anything but observability (it already
  must not — `brain.rs:9-15`).

## Immediate next-session start

1. **D1 first** — add the `spt daemon brain` hidden entry + flip `spt daemon
   run` from `thread::spawn(broker.serve)` to broker-process + spawned brain
   child over the existing socket IPC. Prove the skeleton (broker survives
   brain kill, PTY child intact) **before** moving any logic.
2. Activate `REQ-HAZARD-BROKER-PROCESS-ISOLATION` doc stage (evidence already
   exists: ADR-0018 + design doc) in the D1 commit; impl/unit follow as D1
   lands real structure.
3. Keep `ensure_running`/`is_running`/`daemon stop` external behavior
   identical throughout — they're the single-instance contract the whole
   fleet depends on; only their *owner process* moves (thread → broker proc).
4. Do **not** convert the idempotent pump cadences (D5 [V4]) — only the pulse
   loop's phase-significant timing.