# M5-D3 — shell hosting machinery (JIT plan)

**Status:** **CLOSED 2026-06-04** — D3a (`d3af7e8`) · D3b (`0745394`) · D3c
(`538becc`) · D3d (`984852e`) · D3e + mock-shell E2E (`1271023`) — all
CI-green-FINAL. Successor: `M5-D4-PLAN.md` (sleep/wake).
D0+D1+D2 complete (REQ-CONSENT-1/2 + REQ-INSTALL-4 `[impl,unit]`, CI-green-FINAL at
`23c22a7`). D3 is the biggest M5 task: an agent mints, drives, and tears down
owner-exclusive shell instances hosted by the daemon — the machinery the mock shell
proves and the OS-notification shell (D8) rides. Authoritative model: **CONTEXT §Shell
model** (~line 216), `docs/MANIFEST.md` §Shell adapters, `docs/STORAGE.md` §Shells in
the layout, ADR-0004 (daemon hosting), ADR-0009 (gate nesting), M5-PLAN §D3.

## Goal

`spt shell spawn <adapter> [--alias]` mints `<adapter>-<n>` under
`owlery/<owner>/shells/`, the daemon broker-launches the manifest's `spawn` binary, the
binary `api bind`s (type=Shell) over a link-token handshake, and the three channels flow:
durable **command** (vocabulary-checked agent→shell), durable 2-way **text+file**
(transfer progress-queryable), ephemeral **sensory** (`api emit`, REST-only, never
spooled, dropped unless the owner is live). Spawn passes the D1 grant-store gates
(`require_approval` floor, `max_instances_per_owner` + `over_cap`); only the owner may
drive; discovery honors `broadcast`; the agent's context knows its shells. Hazards 7.4 +
1.3 activate on the daemon's new fan-out.

## Decisions (locked)

- **Layout via the single path source (KH 6.1):** new resolvers in `spt_store::perch` —
  `resolve_shell_perch_path(owner, shell_id)` → `owlery/<owner>/shells/<adapter>-<n>/`
  and `list_shell_instances(owner)`. A third namespace beside flat-Self and `nested/`;
  nothing composes shell paths by hand. Shell ids never collide with Psyche/Worker
  classification (shells are not perch *kinds* — they live under their own `shells/`
  namespace and are resolved only through the shell resolvers, never
  `resolve_perch_path`).
- **`ShellInfo`, not `InfoJson`:** the shell perch `info.json` carries exactly
  `type=Shell, owner, adapter_name, status(online|offline), alias?` (STORAGE.md). A
  dedicated serde struct (`spt_store::shellinfo`) keeps the agent-perch readers
  (liveness, registry stale-clean, consent MRA scan) blind to shells by construction —
  capabilities resolve from the manifest by `adapter_name`, never duplicated on the
  perch.
- **Canonical id minting:** `<adapter>-<n>`, smallest free `n` over **existing dirs**
  (online + offline both occupy; `teardown` frees the slot — the same all-existing rule
  the cap counts). Alias = owner-unique runtime overlay on `info.json` (spawn `--alias` /
  `rename`); alias and canonical id interchangeable everywhere a shell ref is taken.
- **Hosting home:** `spt-daemon::shellhost` owns instance lifecycle (spawn the filled
  `spawn` template as a supervised child, kill on link-break with `pre_close` +
  `close_timeout_ms` at D4a; D3 delivers launch/teardown). The shell binary is a plain
  supervised child — **not** a PTY session (shells talk channels, not terminals); reuse
  `spt_runtime::fill_template` + the detached-spawn machinery, `{link_token}` from the
  catalog (`SUBSTITUTION_KEYS` already lists it).
- **Local-link handshake (CONTEXT §local-link authentication):** spawn mints a per-link
  token (the `{link_token}` substitution); `api bind <shell-id> --type shell --link
  <token>` proves it. The bind derives the link key from the token; every subsequent
  machinery call on the link (`api emit`, `api poll`, stdin frames) carries
  token-derived auth — same ethos as the perch capability token (`api::auth`), shell
  flavor. Channel *encryption* uses the token-derived key on the local frames; Iroh
  already covers cross-node.
- **Command delivery = the agent inject-input modes, reused:** `command_receipt` picks
  HTTP / stdin-via-broker / child-relay. D3 delivers **relay (`api poll`)** as the
  proven path (mock shell uses it — it is the PollListener machinery we already trust)
  **+ stdin-via-broker** (rides the broker `send_effect` input seam). The HTTP mode
  lands when the first HTTP-receipt shell exists (D8's OS-notif shell will choose; if it
  picks http we build it there) — refusing an unimplemented mode at spawn is explicit,
  never a silent fallback.
- **Channels over the shell spool:** the shell perch's `spool.db` (existing
  `spt_store::spool` opened at the shell perch path) holds **command** rows (agent→shell,
  drained by the delivery mode) and **text+file** rows (2-way). Command payloads are
  **vocabulary-checked** against `[shell.capabilities]` (op + arg names) before spooling
  — the toolset bounds what an agent may ask (CONTEXT §binary-trust). File transfer
  reuses `spt_store::xfer` progress (`XferProgress` read/write) so transfers are
  progress-queryable. **Sensory never touches the spool:** `api emit --type <t>`
  (type ∈ `[shell.sensory].types`) delivers to the owner's **live** session via the
  existing live-delivery path and is **dropped with a diagnostic** when the owner isn't
  live — no buffering, by definition.
- **Gate order at spawn (ADR-0009 nesting holds):** owner/visibility scope →
  registered-adapter check (D2 `registry::registered`) → **cap** (`max_instances_per_owner`
  count of all existing instances; at cap `over_cap`: `reject` refuses, `approve` =
  per-spawn approval that never persists) → **approval** (`require_approval` floor:
  `none` proceeds; `remembered` → `grants::check(CAP_SPAWN_SHELL, owner, node,
  Some(adapter))`, allow-always persists the grant; `always` prompts every spawn,
  allow-always suppressed — never writes). Escalation rides D1b's
  `produce_escalation_notif`/`apply_escalation_answer` unchanged.
- **Discovery is local-node in D3:** own instances (alias · canonical id · status) +
  instantiable adapters = registered-on-this-node shell adapters with `broadcast ∈
  {subnet, same-node}`; `none` never surfaces. The *other-node* `broadcast=subnet` arm
  rides the registry distribution and lands with cross-node link (D8c) — the discovery
  fn takes the registered set + (later) registry rows so the seam is the signature, not
  a rewrite.
- **Scheduling (KH 7.4):** every shell instance's host loop runs on its **own thread**
  off the daemon's control loop (same isolation stance as the per-peer pumps); one hung
  shell binary cannot stall another agent's tick or the broker. Boot-time stale
  `index.lock` sweep (KH 1.3) lands in daemon boot for the git-touching runtimes it
  hosts (BranchStore gitrun roots).

## Pieces (build order — one CI-green slice each)

1. **D3a — identity + perch CRUD** (`REQ-EP-5` impl/unit begins).
   `spt_store::perch` shell resolvers + `spt_store::shellinfo::ShellInfo`; mint logic
   (`<adapter>-<n>` smallest-free, alias uniqueness per owner); `spt shell
   spawn/list/teardown/rename` CLI over it (spawn = mint + perch + record — broker
   launch arrives D3b; `relink` lands D4 with the lifecycle, stubbed refusing now);
   owner-ref resolution (alias ↔ canonical). Spawn:create :: relink:online stated in
   help text.
2. **D3b — broker launch + bind handshake** (`REQ-SHELL-1` impl begins).
   `spt-daemon::shellhost` (per-instance host thread: fill `spawn` w/ minted
   `link_token`, supervise child, teardown kills); `api bind --type shell --link`
   (flips `status=online`, derives link key); link-auth required on subsequent
   machinery calls; non-owner / bad-token refused.
3. **D3c — the three channels** (`REQ-SHELL-1` completes impl/unit).
   Command spool rows + vocabulary check + relay (`api poll`) and stdin delivery;
   text+file 2-way rows + `xfer` progress wiring; `api emit --type` sensory → owner's
   live session, never spooled, dropped-with-diagnostic when owner not live; `spt shell
   cmd <ref> <op…>`.
4. **D3d — gates + discovery + context injection** (`REQ-EP-5` gates+discovery).
   Spawn gate chain (cap → over_cap → require_approval through real `grants::decide`,
   qualifier = adapter name; `always` never persists); owner exclusivity on every
   driving surface; discovery fn (own + instantiable per broadcast) feeding `shell
   list` + the agent context injection (the hidden-context/SessionStart seam the notif
   resurface uses).
5. **D3e — hazard activation sweep** (`REQ-HAZARD-DAEMON-SCHED-NONBLOCKING`,
   `REQ-HAZARD-STALE-INDEX-LOCK`).
   7.4: per-instance threads proven — a deliberately-hung mock shell stalls nothing
   else (test with two owners, one hung shell, other agent's tick proceeds). 1.3:
   daemon boot sweeps stale `.git/index.lock` under its hosted git-touching runtime
   roots. Re-assess 2.2 (`REQ-HAZARD-STDIN-SESSION-ID`): the shell api path passes ids
   as argv, not stdin — expect "leave inactive, update rationale" unless D3b grew one.
6. **Mock-shell E2E** (closes D3 acceptance, rides whichever slice completes last):
   spawn → bind+handshake → command → sensory → teardown against a test mock-shell
   binary (contract_e2e helper pattern); non-owner refused; `remembered`/`always` +
   `over_cap` through real grants; alias addressing; `--no-default-features` lib build
   clean.

## THIS slice — D3e (hazard activation sweep + mock-shell E2E — closes D3)

**Locked decisions (researched against the codebase as shipped):**

- **1.3 stale `index.lock` boot sweep** (`REQ-HAZARD-STALE-INDEX-LOCK` → activate
  `[impl, unit]`): a free fn in `spt_store::branchstore` —
  `sweep_stale_index_locks(git_dir) -> Vec<PathBuf>` — removes locks that are
  **0-byte AND mtime > 60s** (the KH 1.3 staleness rule; live locks stay), scanning
  `<git_dir>/index.lock` + `<git_dir>/worktrees/*/index.lock` (worktree locks live
  in the git-dir, not the checkout). Call site: early `Daemon::run()` (daemon.rs
  ~85, after config load, before the broker bind) against the context store's
  `tracked/.seed.git` — path composed via the existing perch/contextstore
  resolvers, **without** opening the full ContextStore on the boot path.
- **7.4 per-instance threads proven** (`REQ-HAZARD-DAEMON-SCHED-NONBLOCKING` →
  activate `[impl, unit]`): the invariant machinery already exists — every broker
  PTY session gets its own drain + exit-waiter threads and every brain connection
  its own handler thread (broker.rs `serve`/`dispatch_spawn`); shell instances ride
  exactly that isolation (stdin-receipt = broker-hosted session; relay = detached).
  D3e tags the thread-spawn sites as the `[impl]` evidence and adds the proving
  test: **two owners, owner A's shell binary deliberately hung** (never reads
  stdin, never writes, never exits), owner B's stdin delivery
  (`deliver_stdin_pending_in`) and a broker control call (`sessions`) complete
  promptly. In-process broker, same harness as `spt-daemon/tests/shellchan.rs`.
  The daemon-hosted **per-agent pulse fan-out** (`run_pulse_loop` × N agents) is
  NOT yet wired into the daemon — that consolidation rides D4+/ADR-0004; the
  hazard's shell-hosting face is what activates here (note this scope in the
  traceable-reqs comment).
- **2.2 re-assess** (`REQ-HAZARD-STDIN-SESSION-ID`): confirmed — the whole shell
  api path passes identity as argv (`--link <token>`, positional shell-id), no
  stdin id anywhere. **Leave `required_stages = []`**; update the rationale comment
  (D3 audited: no stdin session-id seam grew) — doc-only.
- **Mock-shell E2E** (closes D3 acceptance; `REQ-SHELL-1` gains `[int]` already —
  this extends the e2e face): a real `mock-shell` binary added to `adapters/mock`
  (second `[[bin]]`) that takes `--link <token>` + `--home <spt_home>` style env,
  then drives the real api surface: `spt api bind-shell --link` → `spt api poll
  <id> --link` (drains the spooled command + writes what it saw to an evidence
  file) → `spt api emit <id> --type … --link`. The E2E test (new
  `crates/spt/tests/shell_e2e.rs`, `contract_e2e.rs` patterns: `start_inproc_daemon`,
  `sibling_bin`, `env!("CARGO_BIN_EXE_spt")`) registers a test-authored manifest
  whose `spawn` template embeds the absolute `sibling_bin("mock-shell")` path, then
  proves: spawn → broker-launch → bind onlines the perch → `shell cmd` round-trips
  through the binary's poll → sensory emit reaches the live owner (PollListener) →
  teardown kills + frees the slot. Plus through the same E2E home: non-owner
  refused; `remembered` + `over_cap` exercised through the real grant store
  (grant row → silent proceed; at-cap reject); alias addressing throughout.
- **`--no-default-features` lib build stays clean** — already CI-gated; the slice
  just must not break it.

### Tests (D3e)
- sweep: a 0-byte stale lock (mtime backdated > 60s) in the seed git-dir AND in a
  worktree git-dir is removed; a fresh lock (recent mtime) survives; a non-empty
  lock survives; missing dirs no-op (fresh home boots clean); daemon boot calls it
  (unit on the fn + the boot call site).
- 7.4: hung-shell isolation — owner A's broker-hosted shell child hangs forever;
  owner B's `deliver_stdin_pending_in` delivers and `sessions` answers within the
  test deadline; the hung session's kill still works (teardown unblocked).
- E2E: the full spawn→bind→cmd→poll-evidence→emit→teardown chain above; non-owner
  refusal; remembered-grant + cap gates through the real store; alias addressing.
- 2.2: no test (stays inactive) — rationale comment updated.

## DONE — D3d (gates + discovery + context injection) — `984852e`

- **Spawn gate chain** in `cmd_shell` Spawn (cap → over_cap → require_approval,
  ADR-0009 order): per-ADAPTER cap via `list_shells` filtered by `adapter_name`
  (online+offline occupy); `over_cap` reject refuses / approve = per-spawn approval
  that never persists; `require_approval` none / remembered
  (`grants::check(CAP_SPAWN_SHELL, owner, node_hex, Some(adapter))`, allow-always
  writes the grant) / always (never consults nor writes — a granted row does NOT
  silence it). Gates are node-scoped: an identity-less node refuses approval-gated
  spawns (non-minting, the access-gate stance).
- **Escalation rides D1b unchanged** — `apply_escalation_answer` gained its first
  consumer: TTY prompt `[o]nce/[a]lways/[d]eny` (`parse_escalation_choice`
  fail-closed; allow-always offered only where persistence is legal); non-TTY
  refuses `CONSENT_PENDING` with remembered mode firing the consent notif
  (`produce_escalation_notif`; body carries the durable grant-add answer).
- **NEW `spt-daemon::shelldisc`** — `discover(owlery, owner, registered)` (own +
  instantiable; broadcast `none`/absent never surfaces; the registered-set
  parameter is the D8c other-node seam), `shell_context` renderer, shared
  `instance_lines`/`instantiable_lines`. `shell list` renders both sections from
  the same fn; reported boundaries (`api boundary` / bind-time
  `resurface_notifs_with`) inject ONE **deferred** shell-context spool row
  (hook-channel background context; live event-stream drain skips it; empty
  discovery injects nothing).
- **Owner exclusivity** proven as negative tests (non-owner resolve/cmd/teardown/
  rename never reach a foreign instance; a foreign link token never drives another
  shell's poll/emit). REQ-EP-5 evidence extended in place; traceable-reqs 134/134.

## DONE — D3c (the three channels)

- `spt-daemon::shellchan` — channel machinery: `vocab_check` (op + positional args
  mapped onto declared arg names; unknown op/extra args refuse naming the vocabulary),
  typed frame composers (`shell_command`/`shell_text`/`shell_file`/`sensory` EVENT
  types, added to the spt-proto taxonomy), `stamp_frame`/`verify_stamped_frame` (the
  per-message link MAC — CONTEXT §local-link authentication), `spool_shell_frame`,
  and `deliver_stdin_pending` (broker `send_effect` per spool row id — exactly-once,
  Applied-acked, marked delivered only after the ack).
- `shellhost` — `find_shell_by_token` (pure "owner from the link" lookup; bind
  refactored over it); `launch_shell` branches on `command_receipt`: relay (default)
  stays the detached spawn, **stdin** spawns broker-hosted (`launch_shell_brokered_in`,
  wide PTY, session labeled `<owner>/shells/<id>`), **http** refuses at spawn —
  never a silent fallback.
- `api poll <shell-id> --link <token>` — the relay drain (token IS the auth; drains
  the shell spool's stamped frames). `api emit <shell-id> --type <t> <payload>
  --link <token>` — sensory: type checked against `[shell.sensory].types`, live-TCP
  to the owner or **dropped with a diagnostic; no spool call exists on the path**.
- `spt shell cmd <ref> <op> [args…]` (vocabulary-checked command channel) and
  `spt shell send <ref> [text] [--file <path>]` (durable text+file channel; file
  lands under the shell perch `files/`, `XferProgress` Recv-role record at the node
  transfers dir — progress-queryable by xfer id). Both spool first, then drain per
  receipt; shell→agent text rides ordinary `spt send <owner> --from <shell-id>`.
- Activate **REQ-SHELL-1 `[impl,unit]`** in `traceable-reqs.toml` in this commit.

### Tests (this slice)
- vocab: unknown op refused naming the vocabulary; extra args refused; positional →
  named mapping; trailing args omitted.
- frames: typed EVENT shapes parse; MAC stamp roundtrips; tampered/cross-link/
  malformed stamps drop.
- command channel: cmd spools a stamped frame; `api poll --link` drains in order and
  empties; wrong/foreign token refused; deregistered adapter refuses the cmd.
- stdin: broker-hosted launch + `deliver_stdin_pending_in` against an in-process
  broker — frames reach the child's stdin (PTY echo), rows marked delivered only
  after Applied, second drain is a no-op, no broker session errs cleanly.
- sensory: bad type refused listing the vocabulary; owner not live → dropped AND the
  owner spool stays empty (never spooled); owner live (PollListener) → the typed
  sensory frame arrives over TCP.
- text+file: text row spools; file copies under `files/`, progress reads back
  `done`, the file frame carries the xfer id.

## NOT in D3 (lands later in M5)
- Sleep/wake, `persistent`/`wake_command`/watcher, link-break close semantics,
  `relink`, ephemeral-vs-persistent divergence, owner cascade, `api owner-shutdown` —
  **D4** (REQ-SHELL-2).
- Presence-resolution upgrade of the escalation target — **D6** (D1b's MRA precursor
  stands).
- Cross-node link/relink + other-node discovery arm — **D8c**; cross-node **spawn**
  deferred past M5 (instantiate-anywhere).
- HTTP command-receipt mode unless D8's shell demands it earlier.
- Shell-binary sandboxing — accepted-risk stance unchanged (CONTEXT §binary-trust).

## Conventions (carried)
- NO `cargo fmt`. Tag `[impl->REQ-…]` / `[unit->REQ-…]` on real evidence in the same
  commit. `traceable-reqs check` from repo root before declaring done.
- Linux CI clippy `-D warnings` is the real gate; `--no-default-features` must stay
  clean.
- Push each slice → watch the **sha-pinned FINAL** conclusion (`gh run list --json
  conclusion`); never push the next slice over an in-flight run (concurrency
  auto-cancel reads as "cancelled").
- Commit trailer: `Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>`.