# ADR-0045: Endpoint teardown authority — one topology-aware primitive behind both `shutdown` and `stop` - Status: accepted (doyle triage 2026-07-19, TEARDOWN-AUTHORITY milestone, W1) - Source: two hertz field RCAs, both doyle code-verified the same day. (1) `endpoint shutdown` reports `Active -> Suspended` while the broker-hosted session, its harness child, and the child `spt api listen` all keep running (native GNU 0.33.0, independently reproduced on HFENDULEAM, defect present in 0.38.1). (2) `endpoint stop` has the same hole, and it composes with `endpoint run`'s correct `ENDPOINT_CREATE_CONFLICT` refusal into a lifecycle dead end with no in-band exit — observed on doyle's OWN live endpoint, recovered only by an out-of-band scoped kill. Kin: ADR-0041 (endpoint lifecycle truth — hosting authority `controllable`), ADR-0044 (controller lease truth — why teardown must claim no controller), REQ-HAZARD-DAEMON-STOP-REAP (the same reap principle, one scope up). ## Context Both local teardown verbs stamp state they never cause. `cmd_shutdown` (cli.rs) is two legs: remove the ready marker, then `cmd_rest(Suspend)`. The Suspend edge fires the echo commune, the shell cascade, and a re-advertise — it never touches a session. The verb even probes broker-session truth first (`has_live_session_honest`) purely to force `from=alive` so it does not lie `NO_EDGE`; it therefore *knows* the session is live, reports the transition, and leaves it running. `cmd_stop` removes the marker, unregisters the address, `terminal_normalize`s, and advertises. Its own comment calls this "a DEFINITIVE death observation" — an observation it fabricates, because nothing kills anything. Three consequences, all field-observed: 1. **Orphaned subtree.** The broker keeps the whole tree: adapter → node → harness → nested resumed harness + MCP children. A direct-child kill would not reach it either; the surviving `spt api listen` is a descendant. 2. **The stamp is not durable.** Post-stop, `info.json` read `status=offline` but `rest_state=active`, `controlled=true`: the *surviving host re-bound after* the unconditional, CAS-less `terminal_normalize(path, None)`. No hardening of the write fixes this — only the reap makes the stamp true. 3. **A lifecycle dead end.** `endpoint run` correctly refuses `ENDPOINT_CREATE_CONFLICT` rather than silently reattaching (the deliberate no-silent-reattach rule). So the survivor is simultaneously what `stop` claims to have killed and what `run` refuses to work around. With a wedged harness every in-band verb is exhausted: stop lies, run refuses, `rc` replays a dead PTY, `rc --take` acquires control over a process that will never answer. Exit requires out-of-band `taskkill`. ### Why the design docs settle this rather than leave it open - CONTEXT.md:33 gives the broker "PTY master fds, the spawned harness child processes" as the un-transferable resources it holds; CONTEXT.md:30 gives the daemon the PTYs for all hosted sessions. CONTEXT.md:628 defines *suspended* as "the harness session is closed". Compose them: the entity that must close the session is the entity that holds it. No adapter can reap a process it did not spawn. - CONTEXT.md:190 calls `endpoint stop` the "soft teardown", but the axis there is **records**, contrasted against `purge` (the hard, full record wipe) — `cmd_stop`'s own comment confirms it ("spool.db and info.json are preserved"). CONTEXT is *silent* on process, and that silence is the gap this ADR closes. Stop is hard on process, soft on records; purge is hard on records. - **REQ-HAZARD-DAEMON-STOP-REAP already ruled the principle**: `daemon stop` reaps the spt-hosted children it spawned, via job object / process group, "not detached-immortal". That was ruled at DAEMON scope and never extended to ENDPOINT scope. An endpoint verb leaving a subtree alive is the same defect class one scope down. - REQ-ENDPOINT-STOP-OFFLINE is written purely as a *marking* requirement. Today's roster-only behavior is therefore spec-faithful — this needs a NEW requirement, not a bugfix against an existing one. ## Invariant A teardown verb never stamps a terminal or resting state it has not caused. For an endpoint whose session is broker-hosted, "stopped" and "suspended" both mean the broker session row is gone and its process subtree is reaped. Where core holds no process, a teardown verb claims nothing about processes. ## Decisions **1. One shared, topology-aware teardown primitive serves BOTH verbs.** Ordered: resolve the endpoint's broker `SessionInfo` → issue a dedicated sid/endpoint-keyed broker kill → reap the whole descendant subtree → await broker-row removal under a bound → only then stamp, unregister the address, and advertise. The stamp is ordered *after* the confirmed reap, which is what makes it durable against decision 2's re-bind. **2. The split is by hosting topology, not by verb.** Gated on hosting authority (`controllable == Some(true)`, ADR-0041) — the same predicate `cmd_bind`'s online-earn and the livehost reconcile already use, so the codebase carries one authority concept, not two. Harness-hosted / external endpoints keep marker + address + status behavior unchanged: core spawned nothing there and cannot reach into the harness's process tree (CONTEXT.md:39). The misleading doc comment claiming marker-removal stops the listener is corrected to say which topology it describes. **3. The two verbs differ ONLY in ceremony and resulting rest intent.** `shutdown` (graceful): echo commune + shell cascade FIRST, then teardown; result `rest_state=suspended`. `stop` (definitive): no ceremony; result `terminal_normalize`. Teardown, reap, bounded await, and postcondition are identical. **4. The kill never depends on harness cooperation** — no graceful-input path, no waiting on PTY EOF, no "ask it to exit first". A wedged host is the design case, not the edge case: it is the situation that produced this ADR. **5. `stop` fires NO echo commune — a deliberate, documented deviation from CONTEXT.md:647.** A literal reading of 647 ("any active → (dormant|suspended) transition fires an echo commune") would demand one, since `stop` crosses that edge. It is ruled out because an echo requires a responsive harness, and `stop` exists precisely to work when the harness is wedged; an echo-with-timeout would re-import the hang the verb exists to break. The cost is real and is not hidden: **`stop` loses the final context delta, which is why `shutdown` remains the preferred verb and `stop` is the escalation.** CONTEXT.md carries the exception inline so it is never "fixed" back. **6. Teardown claims no controller.** The kill is a dedicated sid/endpoint-keyed broker request, NOT `Brain::attach()` + `kill_session()` — the latter operates on the connection's attached session, which under ADR-0044's lease ladder makes teardown a controller theft with a generation bump behind it. The request is idempotent: an unknown session is a benign success, so replay stays clean. **7. On reap timeout, do NOT stamp cold.** Exit non-zero and leave the state honest. A cold row over a live process is the worse lie — it hides a running subtree behind a stopped row, which is the shape that produces the orphan leaks already paid for twice. Because `stop` is the last rung of the ladder with no in-band escalation behind it, its failure line names the surviving ROOT PID and states the scoped-kill remedy explicitly. **8. The dead end is the acceptance criterion.** After a `stop`, `endpoint run` on the same id must SUCCEED (spawn or resume) and must never answer `ENDPOINT_CREATE_CONFLICT` about a session that `stop` claimed to end. This single assertion is the whole user-visible point of the change. **9. `reconcile_hosted_liveness` stays the catch-up net** for partial failures — it already offlines perches whose broker session is gone. Teardown does not duplicate it. ## Consequences - `spt endpoint stop` becomes a real process teardown for broker-hosted endpoints. This is a behavior change to a public verb: callers who relied on stop as a roster-only operation while keeping the session alive lose that (nothing documented ever promised it, and `run`'s conflict refusal made it unusable in practice). - Recovery from a wedged hosted agent no longer requires out-of-band process killing — the field procedure that recovered doyle becomes the verb's own behavior. - The existing unit `shutdown_soft_stops_and_suspends` asserts marker removal plus stored intent only, i.e. it asserts the bug. It is rewritten, not extended. - `stop` on a wedged host now costs the final context delta (decision 5). Documented at the verb, not just here. ## Amendment 1 — the liveness oracle is MILESTONE-WIDE, not teardown-private (W2, 2026-07-19) Decision 7 says a teardown refuses only over a SURVIVING HOST. W1's gate found that "the broker row is still present" is a PROXY for that, not the thing, and W1 therefore routed its own liveness question through the OS process table (`teardown.rs::root_provably_gone`). That fix was keyed on the path we were standing on rather than on the question being asked, and it left a second answer to the same question in the tree. **Amendment: "does this pid still exist" has ONE answer in spt-core, and it is derived from the OS process table.** `spt-daemon/src/broker.rs::session_is_zombie` computes `wrapper_alive` from `spt_store::proc::is_process_alive`, which is unsound for this question — see the invariant below — so `zombie_verdict` is fed a lie on the platform we primarily ship. `zombie_verdict` itself is correct and stays PURE and unchanged. **The discriminator (todlando, W2 seam read).** `is_process_alive` is unsound exactly when THE ASKER — or a live ancestor of it — still holds an open HANDLE to the target: that is the only condition under which Windows' `OpenProcess` keeps succeeding after termination. Dropping a `Child` closes the handle, so a spawner that drops is honest and a spawner that retains is not. Two consequences that make this checkable rather than a judgement call: the dangerous question is only ever "is it GONE" asked about a process you OWN, and **the same call is sound in the CLI and unsound in the daemon for the very same pid.** **REJECTED ALTERNATIVE — `proc::reap_if_child` before the probe.** Recorded here rather than in a code comment, because a rejected alternative that lives only next to the code gets re-proposed by the next person who never reads that file. It fails twice over: (a) it is a NO-OP on Windows (`proc.rs`: `#[cfg(windows)] let _ = pid;`) — it closes a unix zombie, not a Windows handle, so the tree's one existing mitigation for this class does not defend the only platform that has the class; and (b) decisively, even a Windows equivalent could not help here, because the broker holds `Arc` — hence the handle — for exactly the sessions this predicate is ever asked about. **The handle is held BY CONSTRUCTION for every pid the question concerns, so no handle-based probe can ever be sound at this site.** The process table is not the better oracle; it is the only sound one. **Empty table is NO KNOWLEDGE.** Carried verbatim from W1's ruling and it binds here too: `process_table()` yields empty where the platform has no snapshot (non-Linux unix). At this site it must resolve to `None` — `zombie_verdict(None)` already answers "no probeable pid — never guess", which is the right answer for "no snapshot" as well. It must NEVER manufacture `Some(false)`, which would mass-classify every live session a zombie: the W1 blocker inverted, at broker scope. **Audit, do not mass-migrate.** `is_process_alive` is correct at most of its call sites and a sweeping rename would bury the real change. W2 adjudicates each daemon-side site with the discriminator above and fixes only those both unsound AND reachable. The sound probe is added as a NAMED SIBLING whose name is the question (`process_exists`), and `is_process_alive`'s own doc states which question it answers and which it does not — because the knowledge existed in-tree since 2026 (documented verbatim in `legacy_resident_sweep_e2e.rs`) and never reached either the API or `zombie_verdict`. It was written where it was DISCOVERED instead of where it is CONSUMED.