# Shell-substrate extensions — JIT milestone plan (M11)

> **STATUS: PLANNED (2026-06-15).** Product of the Gateway grill (operator + `doyle`,
> 2026-06-11; CONTEXT.md shell-model mints @ that grill). Grounded against the live
> tree post-v0.7.1 + the CONTEXT.md "Shell model (detailed)" + "Consent & security
> gates" sections (authoritative for meaning). No GSD — hand-authored JIT plan, heir
> to `M5-PLAN.md` (the shell *mechanism*) and `DIGEST-MILESTONE-PLAN.md`. Executes on
> operator/doyle dispatch (→ todlando); `doyle` gates each wave against the RECORDED
> design.
>
> **Roadmap placement: M11** (ROADMAP §"Queued after M10"). M12 (remote attach `spt
> rc`) already shipped (v0.7.0) ahead of this — M11 was deferred behind the
> adapter-enabling work; this is the shell-substrate completion pass.

---

## 1. Goal

The M5 shell *mechanism* shipped (perch, three channels — command/text+file/sensory,
spawn gates, sleep/wake, cross-node link, proven by the mock shell + the real
`spt-shell-notify`). M11 adds the **four substrate extensions the Gateway grill
ratified** so the model is complete enough for the *driven-surface* class of shell
(GameRobot continuous control, usbip opaque protocols, Gateway-owned surfaces):

1. **Drive channel** (owner→shell, REST-only, never-spooled, **latest-wins**) — the
   owner→shell mirror of sensory: continuous real-time control (scroll/crank/stick/
   avatar movement) where a missed frame is superseded by the next, and spooled replay
   of stale control on relink would be *actively wrong*.
2. **Shell tunnel** — a long-lived, **reliable-ordered** opaque byte stream (a
   dedicated QUIC stream pair bound to the link) the channel taxonomy must NOT
   reinterpret; first consumer is usbip URB traffic.
3. **Per-capability approval gates** — the `require_approval` enum riding *individual
   capability entries* (gating the dangerous *act*, not just spawn), with an optional
   **class key** so grants scope finer than the capability (usbip `attach` per
   device-class).
4. **Gateway-as-owner audit** — prove (in code + an E2E) that shell ownership is
   **not agent-exclusive**: any non-Shell endpoint type (Gateway first) may own,
   spawn, drive, command, and link a shell. Close any hardcoded agent-owner assumption.

**End state:** a mock driven-surface shell adapter, owned by a **Gateway-typed**
endpoint, receives latest-wins drive frames (offline ⇒ dropped, never spooled), holds
a reliable-ordered tunnel that round-trips opaque bytes the envelope never touches,
and refuses a class-keyed `attach` until a per-(owner × class × node) grant is written
— all through the real `spt` binary, no downstream shell code, proven on the rig.

## 2. Requirements (register first — rule 3; activate per-wave at evidence — rule 5)

New reqs to **add to `traceable-reqs.toml` with `required_stages = []`** as Task 0,
then activate per wave as evidence lands:

| REQ | Scope | Stages (activate at wave) |
|-----|-------|---------------------------|
| **REQ-SHELL-3** | Drive channel: owner→shell REST-only, never-spooled, latest-wins (supersede, no replay); delivered to the online binary only, dropped-with-diagnostic if offline; manifest `[shell.drive]` vocab; `api`/CLI drive surface; cross-node ride over the ephemeral link (never the durable spool) | `doc, impl, unit, int` |
| **REQ-SHELL-4** | Shell tunnel: dedicated reliable-ordered QUIC stream pair bound to the link, opaque (not enveloped, not taxonomy-interpreted, not spooled); link lifecycle governs it (link-break closes); manifest opt-in; on-LAN posture documented | `doc, impl, unit, int` |
| **REQ-CONSENT-3** | Per-capability approval gates: `require_approval` on a `ShellCapability` + optional `class_key`; grant qualifier = `(owner × class × node)`; reuses the grant store + interactive escalation + tighten-only floor; spawn-gate vs act-gate distinction | `doc, impl, unit, int` |
| **REQ-SHELL-5** | Owner type-agnostic: any non-Shell endpoint type may own/spawn/drive/command/link a shell (CONTEXT §"Owner-linked, exclusive" — ratified 2026-06-11); owner-exclusivity keys on `owner endpoint_id`, never on endpoint *type* | `doc, impl, unit, int` |

These extend, not replace, **REQ-SHELL-1/2** (mechanism) and **REQ-CONSENT-1/2**
(grant store + escalation), all green from M5.

## 3. Design (settled — CONTEXT.md §Shell model + §Consent; do not re-litigate)

Load-bearing points the tasks depend on (CONTEXT line refs):

- **Four channels, not three** (CONTEXT:256-260). Drive is the **owner→shell mirror of
  sensory**: REST-only, never spooled, ephemeral, dropped if the shell is offline, a
  missed frame superseded by the next. Commands = discrete + durable; drive =
  continuous + ephemeral. The existing `compose_sensory_frame` / `api emit` REST path
  is the template — drive reverses the direction (owner→shell) and adds latest-wins
  supersede (no queue, single live slot).
- **Tunnel is distinct from channels** (CONTEXT:262). Long-lived reliable-ordered byte
  stream, dedicated QUIC stream pair on the link, opaque end-to-end. NOT spooled, NOT
  enveloped (carve-out sibling to the `api poll --link` raw-frame exemption, ADR-0020).
  Link-break closes it. Reliable-ordered is the point and the cost → congestion
  surfaces as lag never loss → **acceptable only on-LAN** (document the deployment
  bound; do not silently allow WAN tunnels to degrade).
- **Per-capability gate** (CONTEXT:283). Same `require_approval` enum, same grant store,
  same interactive escalation, same **floor (tighten-only)** semantics as the spawn
  gate. A capability's **class key** scopes the grant qualifier finer than the
  capability id: a remembered HID-class `attach` grant never authorizes a storage-class
  attach. **Spawn gates govern existence; capability gates govern acts.** The grant
  `decide`/`EscalationAsk` plumbing (`spt-daemon::grants`) already carries a *qualifier*
  — the class key maps onto it; no new grant table.
- **Owner is type-agnostic** (CONTEXT:264). Ownership is NOT agent-exclusive — any
  non-Shell endpoint type may own (Gateway the named first). Control-exclusivity is
  *who commands it* (`owner endpoint_id`), not *what type* the owner is. The shell-link
  wire already names the owner whose tree is searched (`shelllink.rs`); the audit
  proves no path keys on the owner's *type*.

**Scope discipline (the watch-item):** drive + tunnel have **no real consumer shell
in-tree yet** (GameRobot, usbip both deferred, `docs/DEFERRED.md`). M11 builds the
**machinery + a mock driven-surface exerciser**, exactly as M5 shipped the shell
mechanism proven by the mock shell + `spt-shell-notify` — NOT a speculative realtime
subsystem with no reader (the M9-PLAN §9 watch-item). The mock exerciser IS the reader.
**This premise is the first thing to gate with doyle** (§6).

## 3a. Gate resolution (doyle PASS — 2026-06-15)

Premise gated PASS, **2 conditions + 1 refinement** folded below:
- **C1 (Q1):** drive + tunnel ship **machinery + mock exerciser ONLY**. No speculative
  GameRobot/usbip real-shell code. The mock exercises every path = the reader (the M5
  model); §7 non-goals hold.
- **C2 (Q2):** tunnel = **same-node mock E2E** (CONTEXT:262 forbids WAN tunnels —
  reliable-ordered ⇒ congestion is lag never loss, on-LAN only). The `[twohost]` WAN
  rung carries **drive + cmd only**; tunnel is documented on-LAN, **not proven
  cross-WAN**.
- **R1 (Q4):** front-load the ownership audit (ex-T4.1, **code-only, NO E2E**) into
  **Task 0**, before W1 — a proof not a change (thesis: ownership already keys on
  `owner endpoint_id`, `shelllink.rs`); if it surfaces any hardcode touching ownership
  signatures, fix it before W2/W3 build drive/tunnel on it. T4.2 (Gateway-typed E2E)
  stays the capstone.
- Q3 (new reqs) — agreed. §5 gate criteria accepted as written; doyle gates each wave at
  evidence-land, `traceable check` EXIT 0 + both-runner green non-negotiable per wave.

## 3b. Execution progress (branch `m11-shell-substrate`, off `main`@cb05600)

- **T0 ✅ @5251d16** — ownership audit (clean: no path gates on owner type; `is_agent_endpoint`/`EndpointFamily::Agent` serve only the user-msg gate + taxonomy) + 4 reqs registered. REQ-SHELL-5 doc/impl activated + green.
- **T1.1 ✅ @6427c7e** — `ShellCapability` gains `require_approval` + `class_key`; `validate()` refuses a class_key without a gated mode; schema regen'd, drift gate clean.
- **T1.2a ✅ @c4ddb57** — act-gate decision core in `linkhost::drive_shell`: `act_gate_decide` (pure) + `act_gate_pending` (I/O front door) + `ShellLinkError::NeedsApproval`. Gated op never spools; class-scoped grants don't cross (storage grant ≠ hid attach); always-mode prompts. 3/3 linkhost unit tests green. REQ-CONSENT-3 still `stages=[]` (tags inert until wave close).
- **⏭ RESUME — T1.2b** (escalation firing + render): (1) serve `run_action` SHELL_LINK_CMD arm — add a distinct `NeedsApproval` outcome that **fires** `grants::produce_escalation_notif` on this (shell's) node + replies `needs_approval` (today the catch-all refuses correctly but silently). (2) Local CLI `cmd_shell` `ShellCmd::Cmd` — catch `ShellLinkError::NeedsApproval(ask)` → fire the escalation notif + print a `CONSENT_PENDING:` line + return non-zero (mirror `escalate_spawn`, cli.rs:5836, for the notif-store/epoch/policy plumbing). `always`-mode suppresses allow-always at the prompt (spawn parity). Answer side already generic (`apply_escalation_answer`).
- **⏭ T1.3** — interactive loopback E2E (mirror `shell_e2e.rs` + the spawn-gate consent test): class-keyed `attach` refuses → escalation → allow-always writes `(owner × hid × node)` → re-`attach` proceeds; a `storage`-class `attach` still refuses. Then **activate REQ-CONSENT-3 = [doc,impl,unit,int]** (doc tag at CONTEXT:283) — activation + int evidence land together.
- Then **W2 (drive) → W3 (tunnel) → W4 (Gateway capstone) → W5 (rig+docs)** per §4. doyle gates each wave at evidence-land.
- **GOTCHA (this session):** HEAD switched main↔branch mid-work once (shared box); T0 was safe on the branch but the working tree briefly showed main. Always `git branch --show-current` before committing M11 work — commit on `m11-shell-substrate`, never `main`.

## 4. Task breakdown

### Task 0 (first commits, pre-W1)
- **T0.1 — ownership audit (code-only, R1).** Grep/trace every shell ownership path
  (spawn, cmd, drive[future], tunnel[future], link, sleep/wake, owner-shutdown) for any
  assumption the owner is an *agent* endpoint; the gate must be `owner endpoint_id`,
  type-agnostic. Fix any hardcode found (expect none). Records the REQ-SHELL-5 `doc`
  finding. NO E2E here. `[doc/impl→REQ-SHELL-5]`
- **T0.2 — register reqs.** Add REQ-SHELL-3/4/5 + REQ-CONSENT-3 to `traceable-reqs.toml`
  with `required_stages = []`; activate per wave as evidence lands.

### Wave 1 — Per-capability approval gates (REQ-CONSENT-3) — lowest risk, no new transport
- **T1.1 — manifest.** `spt-runtime`: add `require_approval: ShellApproval` +
  `class_key: Option<String>` to `ShellCapability`; `validate()` (class_key only with a
  gated approval mode); regenerate `manifest.schema.json` + the drift gate. `[doc/impl/unit]`
- **T1.2 — the act-gate.** `spt-daemon`: gate `shell cmd`/command-channel delivery on the
  per-capability `require_approval`, qualifier = `(owner × class_key × node)`, through the
  existing `grants::decide` + `EscalationAsk` (the M5 spawn-gate consumer is the template);
  honor the tighten-only floor (node/endpoint setting may demand approval the manifest
  set `none`). `[impl/unit]`
- **T1.3 — E2E.** Loopback: a class-keyed `attach` refuses → escalation → allow-always
  writes the `(owner × class × node)` grant → re-`attach` auto-allows; a *different*
  class refuses (grant doesn't cross classes). `[int]`

### Wave 2 — Drive channel (REQ-SHELL-3) — mirror sensory, add latest-wins
- **T2.1 — manifest + frame.** `[shell.drive]` vocab (the drive payload types, sibling of
  `[shell.sensory]`); `EVENT_TYPE_DRIVE` + `compose_drive_frame` (owner→shell direction).
  `[doc/impl/unit]`
- **T2.2 — latest-wins delivery.** `spt-daemon::shellchan`: REST-only owner→shell delivery
  to the **online binary only**, **single live slot** (a new frame supersedes an
  undelivered one — no spool, no queue), dropped-with-diagnostic if the shell is offline
  (mirror the sensory drop-diagnostic; assert NO spool call exists on the path). `api drive`
  / `spt shell drive <id> <payload>` surface. `[impl/unit]`
- **T2.3 — cross-node.** Drive rides the **ephemeral link** (REST class), never the durable
  shell spool — a cross-node drive to an offline shell drops, same as local. `[impl/unit]`
- **T2.4 — E2E.** Mock driven-surface shell: owner drives N frames, only the latest
  observed (supersede proven); owner offline-then-drive drops (no replay on relink). `[int]`

### Wave 3 — Shell tunnel (REQ-SHELL-4) — new reliable-ordered substrate
- **T3.1 — manifest opt-in.** `[shell.tunnel]` (enable flag + an opaque-protocol label for
  diagnostics); `validate()`. `[doc/impl/unit]`
- **T3.2 — tunnel substrate.** (doyle design-gate GO, 2026-06-15.) A **single** (Q1)
  reliable-ordered QUIC stream pair on the **D4 net-stream substrate** (`NetHost::open_stream`,
  broker-owned, survives brain restart, ADR-0004 §B) — a NEW held-stream lifecycle, distinct
  from the one-shot request/reply `serve_shell_link`. W3 anchor = the NetHost **loopback**
  conn (C2 same-node mock; cross-node real-Iroh is the W5 rung, R2). **Open** at shell-online
  (`shellhost::bind_shell_by_token`, manifest-enabled only); broker holds the `stream_id` in
  shell state **online-generation stamped** — *the* crash-proof no-stale-stream leg (R1):
  a stale stream_id from a prior online-gen is NEVER reused after relink even if
  `close_shell`'s drop never ran (brain crash mid-close). **Close** at `close_shell`
  (link-break) = finish+drop = prompt cleanup, NOT the guarantee. **Opaque relay**: tunnel
  bytes ARE the payload (no envelope, no MAC-frame parse, no spool) — sibling carve-out to
  the `api poll --link` raw exemption (ADR-0020). **Surface** (Q2/Q3): dedicated
  `spt shell tunnel <id>` (owner write-in + subscribe-out) + a NEW raw-duplex **`api tunnel`**
  shell-side verb (NOT the child-relay `api poll` — poll is the durable envelope/spool drain;
  conflating them reinterprets opaque bytes). Document + assert the **on-LAN posture**.
  `[impl/unit]`
- **T3.3 — E2E.** Mock tunnel shell (same-node): opaque bytes the envelope grammar would
  mangle (raw binary incl. `<EVENT`-looking bytes) round-trip byte-exact through the tunnel;
  link-break closes the tunnel cleanly; **R1 crash-proof proof** — open → break → relink
  (fresh online-gen) → the old `stream_id` is inert/unreferenced. `[int]`

### Wave 4 — Gateway-as-owner capstone (REQ-SHELL-5) — E2E (audit moved to Task 0, R1)
- **T4.2 — E2E.** A **Gateway-typed** owner (the `gateway_e2e.rs` mock-gateway pattern)
  spawns a mock shell, commands it, drives it, opens a tunnel, and links cross-node —
  all honored exactly as an agent owner. `[unit/int]`

### Wave 5 — Rig proof + docs + close
- **T5.1 — `[twohost]` rung.** Add an M11 ladder rung to `tests/twohost.rs`: cross-node
  Gateway-owned shell — class-gated **cmd + latest-wins drive** only (C2: tunnel is NOT
  proven cross-WAN — same-node mock E2E in T3.3 carries it), HFENDULEAM ↔ kitsubito.
  - **W2 CARRY (doyle gate, 2026-06-15):** the cross-node DRIVE drop-not-wake rung was
    already landed in `tests/twohost.rs` at W2 (T2.4 @13fa53e) but is `[twohost]`-gated, so it
    SKIPS normal CI — its real-Iroh wire (drop-not-wake over the link, never woken/spooled) is
    only proven when the `[twohost]` rig RUNS. W5 MUST fire the `[twohost]`-tagged run and assert
    that rung green (the cross-node LOGIC is unit-proven by `run_action_drive_drops_offline_without_wake_or_spool`,
    but the wire is not). Do not let it ride silently skipped to close.
  - **W3 CARRY — TUNNEL cross-node-on-LAN rung (doyle gate R2, 2026-06-15):** W3's T3.3
    same-node loopback E2E proves the tunnel **MECHANISM** (held-stream lifecycle + byte-exact
    opaque relay + link-break close) — a correct hermetic gate, but NOT the cross-node wire.
    The tunnel's real consumer is **cross-node-on-LAN** (usbip URB across nodes). That real-wire
    proof rides the SAME D4 substrate over a real **Iroh** conn (not loopback) and MUST be a W5
    `[twohost]` rung beside the drive rung — `open_stream` over the real link, opaque bytes
    round-trip byte-exact, link-break closes. **Never ship the tunnel as "cross-node proven" off
    the loopback mock.** W3 docs/claims must read: *"mechanism proven same-node; cross-node-on-LAN
    is the W5 `[twohost]` rung."* (This refines §3a-C2: C2 forbids cross-**WAN** tunnels;
    cross-node-on-**LAN** is in-posture and gets the rung.)
- **T5.2 — docs.** Shell getting-started + manifest reference: the fourth channel, the
  tunnel, per-capability gates, owner-type-agnostic. Version refs not milestone codes in
  the published docs (the public-docs rule). Tunnel docs frame it *"mechanism proven
  same-node; cross-node-on-LAN is the W5 `[twohost]` rung"* (R2, no overclaim). `[doc]`
  - **ADR-0020 cross-ref (doyle gate, 2026-06-15; light, NOT a new ADR):** add the **shell
    tunnel** to ADR-0020's `<EVENT>`-exemption enumeration (ADR-0020:71-74 already exempts the
    shell-command channel) as a sibling carve-out, so a future reader never flags opaque tunnel
    bytes as an ADR-0020 violation. The CONTEXT:262 `[doc->REQ-SHELL-4]` tag already states the
    tunnel is not enveloped/MAC-framed/spooled — this just enumerates it where the exemption lives.
  - **W5 CARRY (doyle, 2026-06-15):** add a `subnet` **how-to topic** (under existing
    REQ-DOCS-6, no new REQ). perri hit `spt how-to subnet` → `NO_SUCH_TOPIC`:
    `HOW_TO_TOPICS` (cli.rs:4692) is v1-locked to `["ready","send"]` (M7 decision 12).
    Add `HOW_TO_SUBNET` text (the create / show-code / join pairing flow + 6-digit code)
    + the registry entry + bump the v1-lock test (the names-vec assertion + the
    topic-content asserts). Subnet/pairing is the highest-value first-run flow and
    currently dead-ends — closes the `ready`/`send` asymmetry.
- **T5.3 — traceable `check` EXIT 0**, full suite green both runners, gate.

## 5. Acceptance / gate criteria (doyle, per wave + capstone)
- `traceable-reqs check` EXIT 0 with REQ-SHELL-3/4/5 + REQ-CONSENT-3 evidenced at the
  activated stages; no commit left activated-but-unevidenced.
- Drive: latest-wins supersede proven; offline-drop proven; **no spool call** on the path.
- Tunnel: byte-exact opaque round-trip (incl. envelope-looking bytes); link-break closes;
  on-LAN posture documented.
- Per-capability gate: class-key grant scopes correctly (doesn't cross classes); floor
  tighten-only honored.
- Gateway owner: E2E parity with an agent owner across all ownership paths.
- Full workspace suite green on hfenduleam + kitsubito; the M11 `[twohost]` rung green.

## 6. Open questions — RESOLVED (doyle PASS 2026-06-15, see §3a)
1. **Build-now vs defer the no-real-consumer machinery.** Drive + tunnel have no in-tree
   consumer (GameRobot/usbip deferred). My read: build the machinery + mock exerciser now
   (M5 precedent — shipped shell mechanism on a mock + notify), the seam M5 explicitly
   deferred concrete shells onto. Confirm this isn't the M9 "realtime subsystem with no
   reader" anti-pattern, or scope drive/tunnel to machinery+mock only (no rig WAN tunnel).
2. **Tunnel on the rig: LAN-only or WAN.** CONTEXT bounds tunnels to on-LAN. Should the
   `[twohost]` rung exercise the tunnel cross-node (the two runners may be same-LAN), or
   keep tunnel to a same-node mock E2E and only drive/cmd cross the WAN rung?
3. **REQ shape.** Mint REQ-SHELL-3/4/5 + REQ-CONSENT-3 as above, or fold per-capability
   into REQ-CONSENT-1/2 with new int + fold owner-type-agnostic into REQ-SHELL-1? I lean
   new reqs (clean evidence, distinct invariants).
4. **Wave order.** Consent (W1) → drive (W2) → tunnel (W3) → Gateway audit (W4) — lowest
   risk first, audit as capstone. OK, or front-load the Gateway audit (it may change
   ownership signatures the other waves build on)?

## 7. Non-goals (explicit)
- Concrete driven-surface shells (GameRobot, usbip binaries) — downstream repos, deferred.
- Shell-binary sandboxing — deferred (DEFERRED.md; disclosed-trust stance holds).
- Cross-node shell **spawn** — stays deferred behind instantiate-anywhere; M11 owner-cross-
  node covers existing-instance drive/cmd/tunnel/link only (the M5-D8c posture).
- WAN-degrading tunnels — out of posture; tunnel is on-LAN by design.