# Subnet mesh milestone — membership seed-proof + roster-only relay

> Status: **DESIGN RATIFIED** (grill-with-docs, 2026-06-08). Authoritative decision = **ADR-0017**; glossary = **CONTEXT.md §Pairing & trust**; requirements = **REQ-MESH-1..6** (minted inactive, activate at execution). This file is the execution-facing plan; ADR-0017 is the *why*.

## The bug it fixes

Non-directly-paired subnet members never see or reach each other — a subnet was the **pairing graph, not a mesh**. Repro (user 2026-06-08): A creates → B joins A → A offline → C joins B → B offline → A online → A and C invisible. Two gaps: pairwise-pin authorization (`trust::is_trusted`) + own-rows-only gossip with no roster/relay (`registryhost::advertise_local`, `peerloop`, `pairing::wire::Frame::Seed = {seed,epoch}`).

## The decided architecture (one paragraph)

**A member = a node that can prove current-epoch seed knowledge.** Authorization moves from a stored pairwise pin to a **live per-connection symmetric seed-proof** at every inbound gate; visibility is restored by relaying a **member roster** (discovery directory) while registry **rows stay own-authored, fetched directly** from each member (push target widens from directly-paired peers to all roster members — a wider *direct* fan-out, never third-party relay). This **preserves KNOWN-HAZARDS 7.5 and 4.10 verbatim** instead of superseding them (no row is ever relayed). Full design + rejected alternative (per-node-signed row relay) in ADR-0017.

## Decisions (resolved in the grill — supersedes the original open-questions)

1. **Scope = full reachability**, not just visibility; the security-critical center is the **inbound-accept gate** (membership predicate replaces `is_trusted` at registry apply, WAN receive, sync, notif, connection accept). → REQ-MESH-1.
2. **Auth = symmetric seed-proof** (not per-node signatures / not row relay). `MK = HKDF(seed, domain ‖ subnet_id ‖ seed_epoch)`; mutual **channel-bound** challenge-response (binds both handshake-proven pubkeys, nonces, subnet_id, seed_epoch, role); **once per connection**, cached on the broker `ConnEntry`; keep-alive keeps connections multi-week so re-proof is restart/partition/rotation-only (no per-message tax); exact-epoch match. → REQ-MESH-1.
3. **Roster subsumes the trust store.** Authorization is live seed-proof; pairwise pinning **retired**. → REQ-MESH-5.
4. **Warn-on-change demoted** to an awareness notice anchored on **machine_id** (not label); fires the REQ-SUBNET-7 re-pair-overwrite event. → REQ-MESH-5.
5. **Hard cutover** — delete `peers.json` + the `is_trusted` path, **no migration** (expendable test fleet re-pairs fresh). → REQ-MESH-5.
6. **Roster** = node-level union-merge grow-set (pubkey, label, machine_id, address, last-seen — **not** the seed), seeded in full at pairing, merged on every member connection under the lease epoch, forgery-inert, persists-through-silence. Removal via **tombstones** (a grow-set can't delete by omission). → REQ-MESH-2.
7. **Rows** stay own-authored; push target = all roster members; **no third-party relay** → 7.5/4.10 preserved. → REQ-MESH-3.
8. **Revoke = seed rotation, timeboxed.** `spt subnet revoke <node>...` (list) tombstones now + schedules **one** rotation at a coalescing window close (default 1h; further revokes coalesce → one epoch bump); `--force-rotate-seed` rotates now (compromised path); seed pushed confidentially over member connections, never in gossip; elevation-gated. → REQ-MESH-4.
9. **Re-seed grace** (auto-heal): prove immediately-prior epoch (N-1) ∧ still on roster → re-seed-only restricted connection; revoked/off-roster denied; ≥2 stale → re-pair. Batch/timebox keeps multi-revoke to one bump so benign offliners stay in the one-deep window. → REQ-MESH-4.
10. **Re-pair after revoke** clears the tombstone for the presented pubkey (deliberate elevated act; a rebuilt compromised node gets a fresh never-tombstoned key). → REQ-MESH-2.
11. **Concurrent liveness probes** in `subnet status --nodes` (bounded by the single-probe ceiling, not k×ceiling — the mesh surfaces all members, many offline). → REQ-MESH-6.
12. **Epoch naming** (no conflation): `seed_epoch` (rotation/revocation) · `lease_epoch` (per-node registry row ordering) · `totp_step` (30s pairing code only).

## Folds in (deferred items, now subsumed)

- Pairing-time hostname capture + post-join address seeding (DEFERRED.md) → roster fields delivered at the ceremony (REQ-MESH-2). Reconcile with REQ-CONV-1 `peer-addrs.json` (the roster's address field may absorb or feed it — planning decides the seam).
- Reconcile with REQ-SUBNET-7 (machine_id re-pair overwrite) — the warn-on-change demotion (REQ-MESH-5) and the tombstone-clear-on-re-pair (REQ-MESH-2) share its machine_id anchor.

## Hazard interaction (the payoff)

- **KH 4.10 preserved** — no transitive **row** gossip (roster relays, rows don't); the eviction lease is untouched. Annotate 4.10's wording accordingly.
- **KH 7.5 preserved** — every row/message arrives from its handshake-proven author; nothing third-party is relayed.
- **New invariants to test** (under REQ-MESH-*): seed-proof channel-bound + mutual (no cross-connection replay); re-seed roster-gated (no revoked re-admit); seed never in roster/registry gossip.

## Test plan

- **int (the acceptance):** the staggered A→B→C repro as an automated multi-daemon test — B offline at the critical step; assert A↔C converge to seeing AND reaching each other. Plus the all-online star (A reaches C with B never relaying). A deterministic repro harness locking the *current* bug is the first artifact (flips green when the mesh lands). → REQ-MESH-3.
- **unit:** seed-proof (valid/forged/wrong-subnet/wrong-epoch/cross-connection-replay/mutual-fail); roster merge lease + union convergence + tombstone (dominate/suppress-reinsert/clear-on-repair) + persist-through-silence; revoke window coalescing (N→1 bump) + re-seed grant/deny truth table + seed-never-in-gossip; gate swap (admits-unpaired-member / rejects-non-member); concurrent-probe timing.

## Migration / fleet

- **Hard cutover, no migration code.** New version deletes `peers.json`; fleet re-pairs fresh. Transient partition during the roll (new seed-proof ↔ old pairwise can't interop) is acceptable (coordinated single-user roll). **Planning verifies the self-update path does not ride member connections** (else the roll deadlocks) — updates are node-local (ADR-0016), expected fine.
- Wire change: `Frame::Seed` gains the roster; rows unchanged. Document the mesh model in CONTEXT (done) + ADR-0017 (done).

## Build phases (D-decomposition)

Each phase ships green and is independently testable; activate the named `REQ-MESH` stage when its phase starts (rule 5). Each `Mesh-D*` gets its own JIT plan when execution reaches it (the project convention — this list is the milestone task layer). **The mesh turns ON at D6** (the staggered harness flips green there).

0. **Mesh-D0 — repro harness (red).** The staggered A→B→C multi-daemon int test (B offline at the critical step; assert A↔C converge to see + reach each other) + the all-online star (A reaches C, B never relays). Lands `#[ignore]`/expected-fail, asserting the *desired* convergence — the acceptance that flips green at D6. First artifact, no production change. *(No REQ activation — it's the REQ-MESH-3 int evidence, tagged when it passes.)*

1. **Mesh-D1 — membership key + seed-proof codec (REQ-MESH-1 impl/unit).** Pure primitive, no wiring: `MK = HKDF(seed, domain ‖ subnet_id ‖ seed_epoch)`; the mutual channel-bound challenge-response frame enc/dec + verify, reusing `pairing::transcript`. unit = MK derivation; valid / forged / wrong-subnet / wrong-`seed_epoch` / cross-connection-replay / mutual-fail. **Activate REQ-MESH-1 (impl, unit).**

2. **Mesh-D2 — seed-proof at connect + ConnEntry flag (REQ-MESH-1 cont).** Run the mutual seed-proof on every dial + accept in `nethost` *before* the connection is usable; cache the membership result on `ConnEntry`; fail → drop. Verify/enable QUIC `keep_alive_interval` < `max_idle_timeout` (open item 1) so connections stay multi-week. Old `is_trusted` gates still run in parallel here (belt + suspenders — no regression; the swap is D5). int = a member connection establishes only when both prove; a non-prover is dropped.

3. **Mesh-D3 — roster store + tombstones + pairing seed-transfer (REQ-MESH-2 impl/unit).** The union-merge grow-set (per-node self-entry under `lease_epoch`, strictly-greater-wins), tombstones (dominate/suppress-reinsert/clear-on-repair), persist-through-silence, atomic/corrupt-degrades-empty — pure, unit-tested. `Frame::Seed` gains the full current roster; the joiner adopts it at pairing. unit = merge lease + union convergence + tombstone truth table + serde additive + full-roster seed on join. **Activate REQ-MESH-2 (impl, unit).**

4. **Mesh-D4 — roster propagation (REQ-MESH-2 cont).** On-connect roster exchange/merge over member connections; addresses captured from the live connection (`conn_remote_addr`). Reconcile with REQ-CONV-1 `peer-addrs.json` (absorb or feed — decide here). int = B knows {A,C}; A connects B, A learns C (roster grows transitively). Still no mesh *reach* yet (gate + fan-out are D5/D6).

5. **Mesh-D5 — gate swap + hard cutover (REQ-MESH-1 gate, REQ-MESH-5).** Swap `is_trusted` → the ConnEntry membership flag at all five inbound gates (registry apply, WAN receive, sync, notif, connection accept). **Delete `peers.json` + the `TrustStore` authorization path** (hard cutover, no migration); demote warn-on-change to the machine_id awareness notice. Directly-paired subset still works (fan-out still narrow until D6) — no regression. unit = gate admits a member / rejects a non-member; no `is_trusted` reference remains in the gate path; machine_id-change emits the notice without blocking. **Activate REQ-MESH-5 (impl, unit).**

6. **Mesh-D6 — fan-out widen → MESH ON (REQ-MESH-3).** Change the registry push target from directly-paired peers to **all roster members** (`peerloop`/`advertise_local`); rows stay own-authored (no relay). **The D0 harness flips green** (A↔C converge). Regression assert: every applied row's author == its handshake origin (no third-party row ever applied). **Activate REQ-MESH-3 (impl, unit, int)** — D0 becomes its int evidence.

7. **Mesh-D7 — revoke + timeboxed rotation + re-seed grace (REQ-MESH-4).** `spt subnet revoke <node>...` (list, elevation-gated): write tombstones (D3) + schedule one coalescing-window rotation (`--force-rotate-seed` immediate); confidential seed push over member connections; force-drop sweep; the re-seed-only restricted handshake (prior-epoch ∧ roster-gated). unit = window coalescing (N→1 bump) + re-seed grant/deny truth table + seed-never-in-gossip + force-rotate skips window. int = revoke → offliner returns → auto-heals on the rig. **Activate REQ-MESH-4 (impl, unit, int).**

8. **Mesh-D8 — status probe concurrency (REQ-MESH-6).** Fan out `subnet status --nodes` serve-probes concurrently (verify open item 2 first — fix if the REQ-SUBNET-5 loop is serial); total wall-time bounded by one ceiling, not k×ceiling. unit = N offline nodes complete within ~one ceiling (mock-clock seam). **Activate REQ-MESH-6 (impl, unit).**

**Pre-D5 check (open item 3):** confirm the self-update path is node-local (ADR-0016), not riding member connections — else the cutover roll could deadlock. Verify before the gate swap lands.

**Build order rationale:** D0 locks the bug; D1→D2 stand up auth without touching gates (safe parallel-run); D3→D4 build discovery; D5 swaps the gate + cuts over (still no new reach); **D6 is the single switch that turns the mesh on** (smallest possible change at the green-flip); D7 adds lifecycle (revoke/rotate/heal); D8 is independent polish. Every phase before D6 is a no-reach-regression; D6 is the reach.