1	# v0.17.0 — robust WAN join + presence-liveness truth — build JIT
2
3	> Next spt-core minor (after v0.16.0 / counter 35). **Design (binding):** ADR-0030 +
4	> `docs/design/robust-wan-subnet-join.md` (the join cluster); the presence fix is a correctness bug
5	> (no ADR). REQs seeded in `traceable-reqs.toml` (`required_stages=["doc"]` for the ADR-0030 cluster,
6	> `[]` for the presence fix). Glossary correction (TOTP-epoch vs TOTP-code) already in CONTEXT.md.
7	> Source: a confirmed field incident this session (SCELTOUIN couldn't join over WAN; and its picker
8	> showed HFENDULEAM's dead endpoints as ONLINE). todlando builds, doyle gates, deployah publishes.
9
10	## Why this milestone (the two confirmed bugs)
11
12	1. **WAN join silently fails on half-broken IPv6.** iroh binds dual-stack; dead-AAAA candidate
13	   attempts silently burn the 30 s rendezvous window. Refuted first: clock/NTP, firewall, relay.
14	   Discriminator: `Test-NetConnection dns.iroh.link -Port 443` IPv6 fails / IPv4 ok. Fixed live by
15	   disabling IPv6. ([[broken-ipv6-poisons-iroh-discovery]])
16	2. **Dead endpoints show ONLINE on remote nodes.** `registryhost.rs:404` advertises a not-alive perch
17	   as `Dormant`; `picker/data.rs:268` maps remote `Dormant → Online`. Dead → Dormant → green ONLINE,
18	   while the home node reads the same perch Offline.
19
20	## Build invariants (every wave)
21
22	- **Activate per wave, int at the fold** (rule 5 / [[traceable-per-wave-activation]]). Tag evidence in
23	  the same commit (`[impl->]`/`[unit->]`/`[int->]` on real code/tests).
24	- **Gate each wave:** `cargo nextest` (spt-net + spt-daemon + spt; NEVER bare `cargo test` on Win —
25	  env races), `cargo clippy --workspace`, `traceable-reqs`/`xtask check`, `xtask gen` where a CLI
26	  surface changed. Pre-kill the orphan `target/debug/spt.exe` scoped to workspace/PID
27	  ([[no-machinewide-killon-shared-runner]]).
28	- **Shared working tree** with todlando — commit SELECTIVELY (`git add <files>`, never `-A`); coordinate
29	  `traceable-reqs.toml` edits (the contended file).
30	- **CI fires on the release PR** (push only triggers main/dev-freeform); twohost rig is `[twohost]`-gated.
31
32	## Waves
33
34	### W1 — per-family bind gate (REQ-NET-FAMILY-GATE) — THE URGENT ROOT FIX
35	- At `NetEndpoint::bind` (`crates/spt-net/src/net/endpoint.rs`), probe each IP family's reachability
36	  ONCE; bind only the working families (dual-stack / IPv4-only / IPv6-only — drop the DEAD family, never
37	  a fixed prefer-IPv4). Re-eval on daemon restart.
38	- `SPT_DISABLE_IPV6` / `SPT_DISABLE_IPV4` overrides force a family off regardless of the probe (mirror
39	  `SPT_NTP_SERVER`).
40	- **Impl-verify:** the iroh single-family bind API (omit the v6/v4 socket) + whether iroh's hole-punch
41	  already fast-fails candidates (may narrow the fix). Keep the probe cheap (short timeout; don't slow bind).
42	- Activate **impl+unit** (unit = family-selection logic both/v4-dead/v6-dead/forced, no live sockets).
43	- *Shippable alone as the urgent WAN-join unblock.*
44
45	### W2 — two-phase meet-before-code join (REQ-JOIN-TWO-PHASE)
46	- Extend `brain.pair_join` (`crates/spt-daemon/src/brain.rs:1009`) into the two-phase stream:
47	  `PairMeetReq{subnet}` → daemon meets `(name, current-epoch)` resolving the seed-holder's **real stable**
48	  pairing address (`pairhost.rs` `connect_seed_holder` / `meet.rs` `dial_via_rendezvous`) → new
49	  `MetMember` event → CLI prompts the code (`cli.rs` `cmd_subnet_join`, `:6236`) → `PairCodeSubmit{code}`
50	  → daemon dials the held addr on `SPT_PAIR_ALPN` + SPAKE2 → `PairJoined`/`PairFail`.
51	- Daemon holds the resolved real-address between phases; **5-minute wait-for-code timeout**. A wrong-code
52	  retry re-runs the **ceremony only** against the held addr (no re-search).
53	- `--code` path stays one-shot (no prompt). Security unchanged (meet is pre-trust `SPT_PAIR_MEET_ALPN`).
54	- Activate **impl+unit+int** (int = real two-phase join over a live meet + wrong-code-retry-hits-ceremony-only
55	  + held-address timeout).
56
57	### W3 — diagnosable join (REQ-JOIN-DIAGNOSTICS)
58	- (a) Live search progress (replace the one-shot "Searching…" `cli.rs:6268` with periodic elapsed/deadline).
59	- (b) Detailed meet-failure (candidates + families + relay-vs-direct + last error) surfaced BEFORE any code
60	  prompt; thread the last error up from `connect_seed_holder` (`pairhost.rs:437`) + `dial_via_rendezvous`
61	  (`meet.rs:281`) (today swallowed).
62	- (c) Propagate the terminal event: `brain.rs:1024` `_ => continue` must deliver `NoSeedHolder`/`PairFail`
63	  to the CLI (this is why the user saw nothing).
64	- (d) `--verbose` / `SPT_LOG` discovery trace (per-probe id, path mDNS/n0-DNS/relay, per-family timeouts).
65	- Activate **impl+unit**. `xtask gen` if the `--verbose` flag touches the CLI surface.
66
67	### W4 — presence-liveness truth (REQ-PRESENCE-LIVENESS-TRUTH) — independent, can parallel W1-W3
68	> Gated vs `resting.rs` + CONTEXT.md: **active/dormant is the multi-instance routing differentiator** —
69	> active = the bare-`id` routing target, dormant = a **warm** non-target sibling ("driving `ling@laptop`
70	> makes `ling@desktop` dormant", resting.rs:97; active→dormant on `AttentionShift`/`Detach`). **suspended**
71	> = cold/resumable-on-wake while its **node is up**. **offline = node DOWN, NEVER self-gossiped** (RestState
72	> has no Offline; a live node only gossips active/dormant/suspended) — remote-inferred via eviction. So
73	> labeling a not-running perch Dormant is the defect; the picker's `Dormant→Online` is *correct* (a warm
74	> dormant instance is an available routing fallback).
75	- **PRIMARY fix — `registryhost.rs:397-405` `else` (not-bound-alive):** live-but-**unbound** (has a live
76	  broker session — `is_perch_alive` is bound-gated; reuse the picker's `is_perch_unbound` signal) →
77	  `Active`/`Dormant` (still warm); **else** (cold, no live session — but its node is UP, since *this very
78	  daemon* is gossiping) → **`Suspended`**, NOT `Dormant` (requires warm) and NOT `Offline` (node is up;
79	  Offline is never self-gossiped). This alone removes the false-ONLINE (picker maps `Suspended` via its
80	  `_` arm → gray).
81	- dormant keeps gossiping (routing/MRA) but **renders as its online flavor** — no distinct glyph (the
82	  dormant→suspended auto-suspend timer disambiguates recency). The display palette is **W5**.
83	- **Flagged (design Q, non-blocking):** `Detach → Dormant` means a *single* detached instance also reads
84	  Dormant; whether an autonomous headless agent should be Active or Dormant is a separate clarification —
85	  does not block this fix.
86	- Activate **impl+unit+int** (int = a dead perch reads NOT-online on a REMOTE node's picker; home-vs-remote agree).
87
88	### W5 — subnet display parity + status palette (REQ-SUBNET-DISPLAY-PARITY) — depends on W4
89	> Grill outcome (operator): local and remote rows render IDENTICALLY; bound/unbound, controlled(+driver),
90	> and harness-only are visible across the subnet. **Fill = actionable** (filled = control-now or wake-on-
91	> live-node; hollow = cannot). Full spec + palette table: `docs/design/subnet-presence-display.md §B`.
92	- **Gossip additive per-`Instance` fields** (`registry::Instance`, forward-compat): `bound`/unbound,
93	  `controlled` + **driver node**, `harness_only`. (Offline never gossiped — node-down is remote-inferred.)
94	- **Remove the remote-reduction** (`picker/data.rs:293-294`) — derive full `EpDisplay` for subnet rows
95	  from the gossiped fields, same path as local.
96	- **`EpDisplay` rework** (`picker/model.rs`): drop `UnboundControlled`; `Unbound` → **red-filled ■**;
97	  `HarnessOnly` → **amber-hollow ▢** (was amber-filled); **add `Suspended` → gray-filled ■**. Picker maps
98	  `Active|Dormant` → online flavor (green/blue/red/amber by bound/controlled/harness), `Suspended` →
99	  gray-filled, `Offline` → gray-hollow (remote-only).
100	- **Desc pane:** a controlled row shows `controlled by <node>`.
101	- Activate **impl+unit+int** (int = a remote row shows unbound/controlled/harness-only/suspended correctly —
102	  local-vs-remote parity).
103
104	### W6 — int activations, docs, release
105	- Activate int stages (W2/W4/W5 + any deferred) at the fold; doyle gates the matrix ×3 both runners (twohost
106	  rig skipped unless a wave touches the cross-host seam — W2's meet/ceremony does NOT change the QUIC subnet
107	  transport, only the pairing handshake order; W5 adds gossip *fields* but not a new transport — judge at the fold).
108	- Public docs (VERSION numbers, no internal codes): gh-pages `networking/overview` + a join/pairing
109	  troubleshooting note (families, `SPT_DISABLE_IPV6`, `--verbose`); the picker **status-square legend** + the
110	  new palette (terminal/picker page); `reference.md` via `xtask gen` if `--verbose` landed. No MANIFEST surface.
111	- Release: bump-in-PR ([[release-standard-bump-in-pr]]), doyle CHANGELOG-vet, sign FRESH, cross-check counter
112	  **36** monotonic from published metadata, deployah publishes.
113
114	## Sequencing / priority
115	- **W1 + W4 are the correctness blockers** (urgent: unblock WAN join + stop false-ONLINE). W2+W3 (join UX
116	  hardening) and W5 (display parity) are enhancements. If a fast patch is wanted, W1 (+W4) ships first as
117	  v0.17.0; W2/W3/W5 fold into the same minor or a follow-up — doyle's call at scope time.
118	- **Two independent clusters** (parallelizable): the **join** cluster (W1-W3, pairing/networking) and the
119	  **presence/display** cluster (W4-W5, registry gossip + picker). W5 depends on W4 (needs the corrected
120	  gossiped Status + the new gossip fields ride the same `Instance` change). Different subsystems from the
121	  join work — todlando can run them as two tracks.
122

# v0.17.0 — robust WAN join + presence-liveness truth — build JIT

> Next spt-core minor (after v0.16.0 / counter 35). **Design (binding):** ADR-0030 +
> `docs/design/robust-wan-subnet-join.md` (the join cluster); the presence fix is a correctness bug
> (no ADR). REQs seeded in `traceable-reqs.toml` (`required_stages=["doc"]` for the ADR-0030 cluster,
> `[]` for the presence fix). Glossary correction (TOTP-epoch vs TOTP-code) already in CONTEXT.md.
> Source: a confirmed field incident this session (SCELTOUIN couldn't join over WAN; and its picker
> showed HFENDULEAM's dead endpoints as ONLINE). todlando builds, doyle gates, deployah publishes.

## Why this milestone (the two confirmed bugs)

1. **WAN join silently fails on half-broken IPv6.** iroh binds dual-stack; dead-AAAA candidate
   attempts silently burn the 30 s rendezvous window. Refuted first: clock/NTP, firewall, relay.
   Discriminator: `Test-NetConnection dns.iroh.link -Port 443` IPv6 fails / IPv4 ok. Fixed live by
   disabling IPv6. ([[broken-ipv6-poisons-iroh-discovery]])
2. **Dead endpoints show ONLINE on remote nodes.** `registryhost.rs:404` advertises a not-alive perch
   as `Dormant`; `picker/data.rs:268` maps remote `Dormant → Online`. Dead → Dormant → green ONLINE,
   while the home node reads the same perch Offline.

## Build invariants (every wave)

- **Activate per wave, int at the fold** (rule 5 / [[traceable-per-wave-activation]]). Tag evidence in
  the same commit (`[impl->]`/`[unit->]`/`[int->]` on real code/tests).
- **Gate each wave:** `cargo nextest` (spt-net + spt-daemon + spt; NEVER bare `cargo test` on Win —
  env races), `cargo clippy --workspace`, `traceable-reqs`/`xtask check`, `xtask gen` where a CLI
  surface changed. Pre-kill the orphan `target/debug/spt.exe` scoped to workspace/PID
  ([[no-machinewide-killon-shared-runner]]).
- **Shared working tree** with todlando — commit SELECTIVELY (`git add <files>`, never `-A`); coordinate
  `traceable-reqs.toml` edits (the contended file).
- **CI fires on the release PR** (push only triggers main/dev-freeform); twohost rig is `[twohost]`-gated.

## Waves

### W1 — per-family bind gate (REQ-NET-FAMILY-GATE) — THE URGENT ROOT FIX
- At `NetEndpoint::bind` (`crates/spt-net/src/net/endpoint.rs`), probe each IP family's reachability
  ONCE; bind only the working families (dual-stack / IPv4-only / IPv6-only — drop the DEAD family, never
  a fixed prefer-IPv4). Re-eval on daemon restart.
- `SPT_DISABLE_IPV6` / `SPT_DISABLE_IPV4` overrides force a family off regardless of the probe (mirror
  `SPT_NTP_SERVER`).
- **Impl-verify:** the iroh single-family bind API (omit the v6/v4 socket) + whether iroh's hole-punch
  already fast-fails candidates (may narrow the fix). Keep the probe cheap (short timeout; don't slow bind).
- Activate **impl+unit** (unit = family-selection logic both/v4-dead/v6-dead/forced, no live sockets).
- *Shippable alone as the urgent WAN-join unblock.*

### W2 — two-phase meet-before-code join (REQ-JOIN-TWO-PHASE)
- Extend `brain.pair_join` (`crates/spt-daemon/src/brain.rs:1009`) into the two-phase stream:
  `PairMeetReq{subnet}` → daemon meets `(name, current-epoch)` resolving the seed-holder's **real stable**
  pairing address (`pairhost.rs` `connect_seed_holder` / `meet.rs` `dial_via_rendezvous`) → new
  `MetMember` event → CLI prompts the code (`cli.rs` `cmd_subnet_join`, `:6236`) → `PairCodeSubmit{code}`
  → daemon dials the held addr on `SPT_PAIR_ALPN` + SPAKE2 → `PairJoined`/`PairFail`.
- Daemon holds the resolved real-address between phases; **5-minute wait-for-code timeout**. A wrong-code
  retry re-runs the **ceremony only** against the held addr (no re-search).
- `--code` path stays one-shot (no prompt). Security unchanged (meet is pre-trust `SPT_PAIR_MEET_ALPN`).
- Activate **impl+unit+int** (int = real two-phase join over a live meet + wrong-code-retry-hits-ceremony-only
  + held-address timeout).

### W3 — diagnosable join (REQ-JOIN-DIAGNOSTICS)
- (a) Live search progress (replace the one-shot "Searching…" `cli.rs:6268` with periodic elapsed/deadline).
- (b) Detailed meet-failure (candidates + families + relay-vs-direct + last error) surfaced BEFORE any code
  prompt; thread the last error up from `connect_seed_holder` (`pairhost.rs:437`) + `dial_via_rendezvous`
  (`meet.rs:281`) (today swallowed).
- (c) Propagate the terminal event: `brain.rs:1024` `_ => continue` must deliver `NoSeedHolder`/`PairFail`
  to the CLI (this is why the user saw nothing).
- (d) `--verbose` / `SPT_LOG` discovery trace (per-probe id, path mDNS/n0-DNS/relay, per-family timeouts).
- Activate **impl+unit**. `xtask gen` if the `--verbose` flag touches the CLI surface.

### W4 — presence-liveness truth (REQ-PRESENCE-LIVENESS-TRUTH) — independent, can parallel W1-W3
> Gated vs `resting.rs` + CONTEXT.md: **active/dormant is the multi-instance routing differentiator** —
> active = the bare-`id` routing target, dormant = a **warm** non-target sibling ("driving `ling@laptop`
> makes `ling@desktop` dormant", resting.rs:97; active→dormant on `AttentionShift`/`Detach`). **suspended**
> = cold/resumable-on-wake while its **node is up**. **offline = node DOWN, NEVER self-gossiped** (RestState
> has no Offline; a live node only gossips active/dormant/suspended) — remote-inferred via eviction. So
> labeling a not-running perch Dormant is the defect; the picker's `Dormant→Online` is *correct* (a warm
> dormant instance is an available routing fallback).
- **PRIMARY fix — `registryhost.rs:397-405` `else` (not-bound-alive):** live-but-**unbound** (has a live
  broker session — `is_perch_alive` is bound-gated; reuse the picker's `is_perch_unbound` signal) →
  `Active`/`Dormant` (still warm); **else** (cold, no live session — but its node is UP, since *this very
  daemon* is gossiping) → **`Suspended`**, NOT `Dormant` (requires warm) and NOT `Offline` (node is up;
  Offline is never self-gossiped). This alone removes the false-ONLINE (picker maps `Suspended` via its
  `_` arm → gray).
- dormant keeps gossiping (routing/MRA) but **renders as its online flavor** — no distinct glyph (the
  dormant→suspended auto-suspend timer disambiguates recency). The display palette is **W5**.
- **Flagged (design Q, non-blocking):** `Detach → Dormant` means a *single* detached instance also reads
  Dormant; whether an autonomous headless agent should be Active or Dormant is a separate clarification —
  does not block this fix.
- Activate **impl+unit+int** (int = a dead perch reads NOT-online on a REMOTE node's picker; home-vs-remote agree).

### W5 — subnet display parity + status palette (REQ-SUBNET-DISPLAY-PARITY) — depends on W4
> Grill outcome (operator): local and remote rows render IDENTICALLY; bound/unbound, controlled(+driver),
> and harness-only are visible across the subnet. **Fill = actionable** (filled = control-now or wake-on-
> live-node; hollow = cannot). Full spec + palette table: `docs/design/subnet-presence-display.md §B`.
- **Gossip additive per-`Instance` fields** (`registry::Instance`, forward-compat): `bound`/unbound,
  `controlled` + **driver node**, `harness_only`. (Offline never gossiped — node-down is remote-inferred.)
- **Remove the remote-reduction** (`picker/data.rs:293-294`) — derive full `EpDisplay` for subnet rows
  from the gossiped fields, same path as local.
- **`EpDisplay` rework** (`picker/model.rs`): drop `UnboundControlled`; `Unbound` → **red-filled ■**;
  `HarnessOnly` → **amber-hollow ▢** (was amber-filled); **add `Suspended` → gray-filled ■**. Picker maps
  `Active|Dormant` → online flavor (green/blue/red/amber by bound/controlled/harness), `Suspended` →
  gray-filled, `Offline` → gray-hollow (remote-only).
- **Desc pane:** a controlled row shows `controlled by <node>`.
- Activate **impl+unit+int** (int = a remote row shows unbound/controlled/harness-only/suspended correctly —
  local-vs-remote parity).

### W6 — int activations, docs, release
- Activate int stages (W2/W4/W5 + any deferred) at the fold; doyle gates the matrix ×3 both runners (twohost
  rig skipped unless a wave touches the cross-host seam — W2's meet/ceremony does NOT change the QUIC subnet
  transport, only the pairing handshake order; W5 adds gossip *fields* but not a new transport — judge at the fold).
- Public docs (VERSION numbers, no internal codes): gh-pages `networking/overview` + a join/pairing
  troubleshooting note (families, `SPT_DISABLE_IPV6`, `--verbose`); the picker **status-square legend** + the
  new palette (terminal/picker page); `reference.md` via `xtask gen` if `--verbose` landed. No MANIFEST surface.
- Release: bump-in-PR ([[release-standard-bump-in-pr]]), doyle CHANGELOG-vet, sign FRESH, cross-check counter
  **36** monotonic from published metadata, deployah publishes.

## Sequencing / priority
- **W1 + W4 are the correctness blockers** (urgent: unblock WAN join + stop false-ONLINE). W2+W3 (join UX
  hardening) and W5 (display parity) are enhancements. If a fast patch is wanted, W1 (+W4) ships first as
  v0.17.0; W2/W3/W5 fold into the same minor or a follow-up — doyle's call at scope time.
- **Two independent clusters** (parallelizable): the **join** cluster (W1-W3, pairing/networking) and the
  **presence/display** cluster (W4-W5, registry gossip + picker). W5 depends on W4 (needs the corrected
  gossiped Status + the new gossip fields ride the same `Instance` change). Different subsystems from the
  join work — todlando can run them as two tracks.


1	# Robust WAN subnet join: meet-before-code + per-family bind gate
2
3	<!-- [doc->REQ-JOIN-TWO-PHASE] [doc->REQ-NET-FAMILY-GATE] [doc->REQ-JOIN-DIAGNOSTICS] -->
4
5	## Status
6
7	accepted (2026-06-27; grilled with operator + `doyle`, `/grill-with-docs`, against a confirmed field incident). Builds on ADR-0002 (networking in core, n0 relay/DNS default), the TOTP-seeded SPAKE2 pairing model (CONTEXT.md "Pairing & trust"), and the rendezvous/meet path (`crates/spt-net/src/net/pairing/{meet,rendezvous,totp,ntp}.rs`, `crates/spt-daemon/src/pairhost.rs`). Full per-item spec: `docs/design/robust-wan-subnet-join.md`.
8
9	## Context
10
11	A remote machine (SCELTOUIN) could not join an spt subnet over WAN: it "never found a machine," then silently stopped — no error. Diagnosis (confirmed by toggling the variable): the host had **half-broken IPv6** (AAAA records resolve, the IPv6 path is dead). spt's iroh endpoint binds **dual-stack** and attempts both families for n0 discovery (`dns.iroh.link`) and the rendezvous hole-punch; the dead-IPv6 attempts don't refuse, they **silently time out**, burning the 30 s rotating-rendezvous window (`SEARCH_DEADLINE` 75 s). Usually a silent timeout; occasionally a slow meet whose pre-entered code had rotated → "Wrong code (codes rotate every 30 s)". Disabling IPv6 fixed it instantly.
12
13	Three defects surfaced behind one symptom:
14
15	1. **A glossary conflation that misshaped the flow.** CONTEXT.md called the **TOTP-code** "the link-discovery selector." It is not — the rendezvous/meet selector is `(subnet-name, TOTP-epoch)` (`rendezvous_token = SHA256(domain ‖ name ‖ totp_step)`), both **public**. The secret **code** is the SPAKE2 password *only*, used in the ceremony *after* the meet. The flow prompted for the code up front because the mental model said "the code routes discovery" — so the entered code aged across a slow meet and went stale.
16	2. **No resilience to half-broken IPv6.** Dead-AAAA candidate attempts silently consume the rendezvous window; the operator had to disable IPv6 by hand.
17	3. **A silent failure path.** `connect_seed_holder` (`pairhost.rs:437`) retries for 75 s with no output; `dial_via_rendezvous` (`meet.rs:281`) discards each probe error; `brain.rs:1024` `_ => continue` can swallow the terminal event — so the user saw nothing.
18
19	## Decision
20
21	### 1. Meet-before-code — a two-phase join
22
23	Because the meet is **code-independent** (selector = public `(name, epoch)`), reorder the join so the code is collected **after** a member is found and used **immediately** at the ceremony — fresh regardless of how long discovery took.
24
25	Extend the `brain.pair_join` event loop into two phases on one stream: CLI `PairMeetReq` → daemon meets `(name, current-epoch)`, resolving the seed-holder's **real, stable** pairing address → `MetMember` event → CLI prompts the code → `PairCodeSubmit` → daemon runs the SPAKE2 ceremony against the held address → `PairJoined`/`PairFail`. The daemon holds the resolved real-address between phases, bounded by a **5-minute wait-for-code timeout**. A genuine wrong-code retry re-runs only the ceremony against the held address — **no re-search**. The non-interactive `--code` path stays one-shot (no prompt); it relies on fast discovery (decision 2) to keep the supplied code in its window. Security is unchanged: the meet is already pre-trust/unauthenticated (`SPT_PAIR_MEET_ALPN`); authentication stays in SPAKE2.
26
27	### 2. Per-family bind gate
28
29	spt-core controls which IP families the iroh endpoint binds (`NetEndpoint::bind`). At bind, probe each family's reachability **once** and bind **only the families that actually work**: dual-stack when both are healthy, IPv4-only when IPv6 is the half-broken kind, **IPv6-only when IPv4 is the dead one** (drop the *dead* family, never a fixed "prefer IPv4" — IPv6-only networks must still work). Re-evaluated on daemon restart. Explicit overrides `SPT_DISABLE_IPV6` / `SPT_DISABLE_IPV4` force a family off (escape hatch + determinism + testing), mirroring the `SPT_NTP_SERVER` pattern.
30
31	### 3. Diagnosable join
32
33	The meet must never fail silently: (a) **live progress** during the search (elapsed / deadline) so silence never reads as a hang; (b) a **detailed failure** on exhaustion — rendezvous candidates and families attempted, relay vs direct, the last concrete error — surfaced **before** any code prompt (a dead subnet must not make the user fetch a code); (c) **propagate the terminal event** (`brain.rs:1024` must deliver `NoSeedHolder`/`PairFail` to the CLI, not `continue` it away); and (d) a `--verbose` / `SPT_LOG` **discovery trace** (per-probe id, discovery path, timeouts) so the next WAN-join issue is self-serve.
34
35	## Considered and rejected
36
37	- **"Find a machine first, then prompt for the code" taken literally as impossible.** Initially thought blocked because the id seemed code-derived; the code proves the selector is the public epoch, so the reorder is sound. Recorded because the conflation was load-bearing.
38	- **Probe-then-join (two stateless calls: quick probe, then a one-shot join with the code).** Simpler/stateless but the code is still entered *before* the join-meet, so a slow second meet re-strands it — it does not actually guarantee the goal. Rejected for the callback-stream (decision 1).
39	- **"Always prefer IPv4."** Breaks IPv6-only networks. Rejected for "drop the *dead* family."
40	- **Shrinking the staleness window via fast discovery alone (keep one-shot).** The IPv6 fix shrinks it but a momentarily slow meet still strands the code. Fast discovery is necessary (and is decision 2) but not sufficient — the reorder is what removes the window.
41	- **Periodic family re-evaluation.** Deferred — once-at-bind (re-probe on restart) is enough for v1; live re-evaluation is added complexity without a demonstrated need.
42
43	## Consequences
44
45	- WAN join is robust to half-broken IPv6 with no manual intervention, and the code can no longer go stale across a slow meet.
46	- A join that fails says **why**, before bothering the user for a code — and `--verbose`/`SPT_LOG` makes the discovery path inspectable (this incident needed code-reading + manual `Test-NetConnection`).
47	- New protocol surface: a `MetMember` event + a `PairCodeSubmit` message + a daemon-side held-address state machine with a 5-minute timeout. The `--code` path and scripted automation are unchanged.
48	- The per-family gate is reusable beyond join — any spt connectivity benefits from not attempting a dead family.
49	- CONTEXT.md is corrected: **TOTP-epoch** (public, routes the meet) vs **TOTP-code** (secret, SPAKE2 only); "the code routes discovery" is now an explicit _Avoid_.
50

# Robust WAN subnet join: meet-before-code + per-family bind gate

<!-- [doc->REQ-JOIN-TWO-PHASE] [doc->REQ-NET-FAMILY-GATE] [doc->REQ-JOIN-DIAGNOSTICS] -->

## Status

accepted (2026-06-27; grilled with operator + `doyle`, `/grill-with-docs`, against a confirmed field incident). Builds on ADR-0002 (networking in core, n0 relay/DNS default), the TOTP-seeded SPAKE2 pairing model (CONTEXT.md "Pairing & trust"), and the rendezvous/meet path (`crates/spt-net/src/net/pairing/{meet,rendezvous,totp,ntp}.rs`, `crates/spt-daemon/src/pairhost.rs`). Full per-item spec: `docs/design/robust-wan-subnet-join.md`.

## Context

A remote machine (SCELTOUIN) could not join an spt subnet over WAN: it "never found a machine," then silently stopped — no error. Diagnosis (confirmed by toggling the variable): the host had **half-broken IPv6** (AAAA records resolve, the IPv6 path is dead). spt's iroh endpoint binds **dual-stack** and attempts both families for n0 discovery (`dns.iroh.link`) and the rendezvous hole-punch; the dead-IPv6 attempts don't refuse, they **silently time out**, burning the 30 s rotating-rendezvous window (`SEARCH_DEADLINE` 75 s). Usually a silent timeout; occasionally a slow meet whose pre-entered code had rotated → "Wrong code (codes rotate every 30 s)". Disabling IPv6 fixed it instantly.

Three defects surfaced behind one symptom:

1. **A glossary conflation that misshaped the flow.** CONTEXT.md called the **TOTP-code** "the link-discovery selector." It is not — the rendezvous/meet selector is `(subnet-name, TOTP-epoch)` (`rendezvous_token = SHA256(domain ‖ name ‖ totp_step)`), both **public**. The secret **code** is the SPAKE2 password *only*, used in the ceremony *after* the meet. The flow prompted for the code up front because the mental model said "the code routes discovery" — so the entered code aged across a slow meet and went stale.
2. **No resilience to half-broken IPv6.** Dead-AAAA candidate attempts silently consume the rendezvous window; the operator had to disable IPv6 by hand.
3. **A silent failure path.** `connect_seed_holder` (`pairhost.rs:437`) retries for 75 s with no output; `dial_via_rendezvous` (`meet.rs:281`) discards each probe error; `brain.rs:1024` `_ => continue` can swallow the terminal event — so the user saw nothing.

## Decision

### 1. Meet-before-code — a two-phase join

Because the meet is **code-independent** (selector = public `(name, epoch)`), reorder the join so the code is collected **after** a member is found and used **immediately** at the ceremony — fresh regardless of how long discovery took.

Extend the `brain.pair_join` event loop into two phases on one stream: CLI `PairMeetReq` → daemon meets `(name, current-epoch)`, resolving the seed-holder's **real, stable** pairing address → `MetMember` event → CLI prompts the code → `PairCodeSubmit` → daemon runs the SPAKE2 ceremony against the held address → `PairJoined`/`PairFail`. The daemon holds the resolved real-address between phases, bounded by a **5-minute wait-for-code timeout**. A genuine wrong-code retry re-runs only the ceremony against the held address — **no re-search**. The non-interactive `--code` path stays one-shot (no prompt); it relies on fast discovery (decision 2) to keep the supplied code in its window. Security is unchanged: the meet is already pre-trust/unauthenticated (`SPT_PAIR_MEET_ALPN`); authentication stays in SPAKE2.

### 2. Per-family bind gate

spt-core controls which IP families the iroh endpoint binds (`NetEndpoint::bind`). At bind, probe each family's reachability **once** and bind **only the families that actually work**: dual-stack when both are healthy, IPv4-only when IPv6 is the half-broken kind, **IPv6-only when IPv4 is the dead one** (drop the *dead* family, never a fixed "prefer IPv4" — IPv6-only networks must still work). Re-evaluated on daemon restart. Explicit overrides `SPT_DISABLE_IPV6` / `SPT_DISABLE_IPV4` force a family off (escape hatch + determinism + testing), mirroring the `SPT_NTP_SERVER` pattern.

### 3. Diagnosable join

The meet must never fail silently: (a) **live progress** during the search (elapsed / deadline) so silence never reads as a hang; (b) a **detailed failure** on exhaustion — rendezvous candidates and families attempted, relay vs direct, the last concrete error — surfaced **before** any code prompt (a dead subnet must not make the user fetch a code); (c) **propagate the terminal event** (`brain.rs:1024` must deliver `NoSeedHolder`/`PairFail` to the CLI, not `continue` it away); and (d) a `--verbose` / `SPT_LOG` **discovery trace** (per-probe id, discovery path, timeouts) so the next WAN-join issue is self-serve.

## Considered and rejected

- **"Find a machine first, then prompt for the code" taken literally as impossible.** Initially thought blocked because the id seemed code-derived; the code proves the selector is the public epoch, so the reorder is sound. Recorded because the conflation was load-bearing.
- **Probe-then-join (two stateless calls: quick probe, then a one-shot join with the code).** Simpler/stateless but the code is still entered *before* the join-meet, so a slow second meet re-strands it — it does not actually guarantee the goal. Rejected for the callback-stream (decision 1).
- **"Always prefer IPv4."** Breaks IPv6-only networks. Rejected for "drop the *dead* family."
- **Shrinking the staleness window via fast discovery alone (keep one-shot).** The IPv6 fix shrinks it but a momentarily slow meet still strands the code. Fast discovery is necessary (and is decision 2) but not sufficient — the reorder is what removes the window.
- **Periodic family re-evaluation.** Deferred — once-at-bind (re-probe on restart) is enough for v1; live re-evaluation is added complexity without a demonstrated need.

## Consequences

- WAN join is robust to half-broken IPv6 with no manual intervention, and the code can no longer go stale across a slow meet.
- A join that fails says **why**, before bothering the user for a code — and `--verbose`/`SPT_LOG` makes the discovery path inspectable (this incident needed code-reading + manual `Test-NetConnection`).
- New protocol surface: a `MetMember` event + a `PairCodeSubmit` message + a daemon-side held-address state machine with a 5-minute timeout. The `--code` path and scripted automation are unchanged.
- The per-family gate is reusable beyond join — any spt connectivity benefits from not attempting a dead family.
- CONTEXT.md is corrected: **TOTP-epoch** (public, routes the meet) vs **TOTP-code** (secret, SPAKE2 only); "the code routes discovery" is now an explicit _Avoid_.


Design clear. Now reading impl sites + REQ seeds + design doc detail.


1	//! The Iroh endpoint — the node's WAN/LAN presence (D1).
2	//!
3	//! [`NetEndpoint`] wraps an `iroh::Endpoint` bound to the node's **own** Ed25519
4	//! identity (`spt-proto`'s [`Identity`]): the iroh `EndpointId` *is* the node's
5	//! public key (same 32-byte seed → same ed25519 keypair), so a node's network
6	//! address and its stable name (`CONTEXT.md` "SptNode") are one and the same — no
7	//! second identity to reconcile. Connections speak the [`SPT_NET_ALPN`] protocol.
8	//!
9	//! Relay policy ([`RelayPolicy`]) is the ADR-0002 v1 relay stance made concrete:
10	//! n0's public relays by default for zero-config onboarding, a self-host escape
11	//! hatch, and a disabled mode (LAN/air-gapped + hermetic tests). See
12	//! [`RELAY_DISCLOSURE`] for the plain-language disclosure that ships to users.
13	//!
14	//! API shape is pinned to iroh 0.98.2 (Spike #2 — the API churns every release).
15	//! This crate owns the *mechanism*; the broker (ADR-0004 §B, D4) owns the *process*
16	//! that holds a live endpoint across brain restarts — [`NetEndpoint::iroh`] is the
17	//! escape hatch the daemon uses to take ownership.
18
19	use std::time::Duration;
20
21	use iroh::address_lookup::MdnsAddressLookup;
22	use iroh::endpoint::{presets, IdleTimeout, Incoming, PortmapperConfig, QuicTransportConfig};
23	use iroh::{Endpoint, EndpointId, RelayMap, RelayMode, SecretKey};
24	use spt_proto::identity::{Identity, PublicKey};
25
26	// Re-exported for the daemon (D4): the broker holds live QUIC connections +
27	// streams in its tables and dialable addresses in its IPC frames without growing
28	// its own iroh dependency edge — the exact version pin (0.98.2) stays in this
29	// one crate.
30	pub use iroh::endpoint::{Connection, RecvStream, SendStream};
31	pub use iroh::EndpointAddr;
32
33	/// The ALPN every spt-core node-to-node QUIC connection negotiates. Versioned so a
34	/// future wire break is a new protocol id, not a silent incompatibility.
35	pub const SPT_NET_ALPN: &[u8] = b"spt-core/net/0";
36
37	/// QUIC keep-alive PING interval — the mesh's "stay warm" knob (REQ-MESH-1).
38	/// A member connection that proves seed-knowledge once at connect must then stay
39	/// open through indefinite idleness, so re-proof is a restart/partition/rotation
40	/// event, never a per-message tax. iroh's default leaves keep-alive **off** and
41	/// idles a quiet connection out at 30s; without this a meshed-but-quiet member
42	/// would silently drop and re-handshake constantly. Must be strictly less than
43	/// [`MESH_MAX_IDLE`] so a PING always lands before the idle timer fires.
44	const MESH_KEEP_ALIVE: Duration = Duration::from_secs(15);
45
46	/// QUIC max idle timeout for spt connections. A live PING every
47	/// [`MESH_KEEP_ALIVE`] resets this timer, so a connection with both ends up
48	/// stays open indefinitely; a real partition or peer death tears it down within
49	/// this bound (then the next dial re-proves membership). Bounded, not infinite:
50	/// a dead peer must eventually leave the broker conn table (the D4c presence seam).
51	const MESH_MAX_IDLE: Duration = Duration::from_secs(60);
52
53	/// The spt QUIC transport config: keep-alive on, bounded idle timeout (see
54	/// [`MESH_KEEP_ALIVE`] / [`MESH_MAX_IDLE`]). Factored out so the keep-warm
55	/// invariant (`keep_alive < max_idle`) is assertable without reading back iroh's
56	/// opaque config type.
57	// [impl->REQ-MESH-1]
58	fn spt_transport_config() -> QuicTransportConfig {
59	    QuicTransportConfig::builder()
60	        .keep_alive_interval(MESH_KEEP_ALIVE)
61	        .max_idle_timeout(Some(
62	            IdleTimeout::try_from(MESH_MAX_IDLE).expect("60s is a valid QUIC idle timeout"),
63	        ))
64	        .build()
65	}
66
67	/// Plain-language disclosure of the soft runtime dependency on n0-operated relay
68	/// infrastructure (ADR-0002 v1 relay stance). Surfaced to the user by the CLI/setup
69	/// path; the dependency is operational + metadata-only (traffic is end-to-end
70	/// encrypted) and escapable via [`RelayPolicy::SelfHosted`].
71	// [impl->REQ-NET-2]
72	pub const RELAY_DISCLOSURE: &str = "\
73	By default spt connects through n0's free public relay servers to find your other \
74	machines and to assist NAT hole-punching. Relays only help peers locate and connect \
75	to each other — your traffic between machines is end-to-end encrypted and the relay \
76	cannot read it. You can run your own relay instead (self-host) or disable relays \
77	entirely for a LAN-only/air-gapped setup.";
78
79	/// Where this endpoint looks for relay assistance (ADR-0002). Maps onto iroh's
80	/// [`RelayMode`]; the default ([`RelayPolicy::N0Default`]) is the zero-config path.
81	// [impl->REQ-NET-2]
82	#[derive(Debug, Clone)]
83	pub enum RelayPolicy {
84	    /// n0's public production relays — the zero-config "just works" default.
85	    N0Default,
86	    /// Self-hosted relay(s) — the principled escape hatch from the n0 dependency.
87	    SelfHosted(RelayMap),
88	    /// No relays: direct/LAN connectivity only. Used by air-gapped LANs and the
89	    /// hermetic loopback tests (no network reached at all).
90	    Disabled,
91	}
92
93	/// Whether the endpoint also runs **mDNS** LAN discovery on top of the preset's n0
94	/// DNS discovery. With mDNS on, a node advertises itself and resolves peers by id
95	/// alone on the local network, so two machines on the same LAN find each other with
96	/// no ticket exchange and no relay round-trip ([`NetEndpoint::connect_by_id`]).
97	// [impl->REQ-NET-1]
98	#[derive(Debug, Clone, Copy, PartialEq, Eq)]
99	pub enum LocalDiscovery {
100	    /// Advertise + resolve over mDNS (swarm-discovery) in addition to n0 DNS.
101	    Mdns,
102	    /// No mDNS — WAN discovery only (and direct addresses). Used by hermetic tests.
103	    Off,
104	}
105
106	/// Which interfaces the endpoint's UDP sockets bind — a mechanical knob, not
107	/// policy. [`BindScope::Loopback`] exists for hermetic tests: a loopback-only
108	/// listener never trips the Windows Defender Firewall "allow access?" prompt
109	/// (the prompt fires per freshly-built test executable that listens on
110	/// `0.0.0.0`), and it makes "no network reached" literally true at the socket.
111	#[derive(Debug, Clone, Copy, PartialEq, Eq)]
112	pub enum BindScope {
113	    /// Bind the unspecified addresses (`0.0.0.0` / `[::]`) — all interfaces.
114	    /// The production scope; required for any off-host connectivity.
115	    All,
116	    /// Bind `127.0.0.1` / `[::1]` only. Hermetic tests: loopback dials work,
117	    /// nothing can reach (or prompt about) a real interface.
118	    Loopback,
119	    /// Bind all interfaces on an **explicit UDP port** (`0.0.0.0:p` / `[::]:p`)
120	    /// instead of an ephemeral one. A node whose port is stable can be named by
121	    /// a static `ip:port` ticket (firewall allow-rules, the two-host rig's
122	    /// env-derived peer addresses) with no relay/discovery infrastructure.
123	    Port(u16),
124	}
125
126	/// Failures binding or dialing the endpoint. Wraps iroh's error chain as a message
127	/// so iroh's churning error types don't leak into this crate's public surface
128	/// (matches the workspace's hand-rolled, dependency-free error convention).
129	#[derive(Debug)]
130	pub enum NetError {
131	    /// The endpoint could not bind its UDP socket / initialise the transport.
132	    Bind(String),
133	    /// Dialing a peer failed (no path, ALPN mismatch, peer down, …).
134	    Connect(String),
135	}
136
137	impl std::fmt::Display for NetError {
138	    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
139	        match self {
140	            NetError::Bind(e) => write!(f, "endpoint bind failed: {e}"),
141	            NetError::Connect(e) => write!(f, "connect failed: {e}"),
142	        }
143	    }
144	}
145
146	impl std::error::Error for NetError {}
147
148	/// A bound spt-core network endpoint: the node's reachable presence on the subnet.
149	pub struct NetEndpoint {
150	    inner: Endpoint,
151	}
152
153	impl NetEndpoint {
154	    /// Bind an endpoint to this node's identity under the given relay policy.
155	    ///
156	    /// The iroh secret key is derived from the identity's 32-byte ed25519 seed, so
157	    /// the resulting `EndpointId` equals the node's [`PublicKey`] — the address and
158	    /// the name are the same value.
159	    // [impl->REQ-NET-1] [impl->REQ-NET-2]
160	    pub async fn bind(
161	        identity: &Identity,
162	        relay: RelayPolicy,
163	        discovery: LocalDiscovery,
164	    ) -> Result<NetEndpoint, NetError> {
165	        Self::bind_with_alpns(identity, relay, discovery, vec![SPT_NET_ALPN.to_vec()]).await
166	    }
167
168	    /// Bind on an explicit ALPN set instead of the default trusted [`SPT_NET_ALPN`].
169	    ///
170	    /// The pairing driver uses this to stand up a **dedicated pre-trust endpoint** bound
171	    /// only to the pairing ALPN (`super::pairing::wire::SPT_PAIR_ALPN`) — the pre-trust
172	    /// accept loop must not share the trusted message surface (ADR-0005; `pairing.rs`
173	    /// "Why a dedicated pre-trust ALPN"). All relay/discovery handling is identical to
174	    /// [`bind`](Self::bind).
175	    // [impl->REQ-NET-1] [impl->REQ-NET-2]
176	    pub async fn bind_with_alpns(
177	        identity: &Identity,
178	        relay: RelayPolicy,
179	        discovery: LocalDiscovery,
180	        alpns: Vec<Vec<u8>>,
181	    ) -> Result<NetEndpoint, NetError> {
182	        Self::bind_scoped(identity, relay, discovery, alpns, BindScope::All).await
183	    }
184
185	    /// The full bind form: [`bind_with_alpns`](Self::bind_with_alpns) plus an
186	    /// explicit [`BindScope`]. Hermetic tests pass [`BindScope::Loopback`] so the
187	    /// endpoint never listens on a real interface (and never triggers a Windows
188	    /// firewall prompt per test binary); everything else delegates here with
189	    /// [`BindScope::All`].
190	    // [impl->REQ-NET-1] [impl->REQ-NET-2]
191	    pub async fn bind_scoped(
192	        identity: &Identity,
193	        relay: RelayPolicy,
194	        discovery: LocalDiscovery,
195	        alpns: Vec<Vec<u8>>,
196	        scope: BindScope,
197	    ) -> Result<NetEndpoint, NetError> {
198	        // iroh and spt-proto both build the ed25519 keypair from the same 32-byte
199	        // seed, so this preserves the node's stable public-key name on the wire.
200	        let secret = SecretKey::from_bytes(&identity.seed());
201	        let builder = Endpoint::builder(presets::N0)
202	            .secret_key(secret)
203	            .alpns(alpns)
204	            // Keep meshed member connections warm (REQ-MESH-1): seed-proof runs
205	            // once at connect, so a quiet connection must not idle out and force
206	            // a re-handshake. Transport-only, orthogonal to BindScope.
207	            .transport_config(spt_transport_config());
208	        // Loopback scope replaces the builder's pre-configured unspecified binds
209	        // (0.0.0.0 / [::]) with loopback-only sockets, AND disables iroh's
210	        // portmapper (UPnP/PCP/NAT-PMP — on by default), which would otherwise
211	        // bind real-interface UDP sockets for its SSDP/NAT-PMP probes: useless
212	        // on loopback, and the other source of per-test-binary Windows firewall
213	        // prompts.
214	        let builder = match scope {
215	            BindScope::All => builder,
216	            BindScope::Loopback => builder
217	                .clear_ip_transports()
218	                .bind_addr("127.0.0.1:0")
219	                .map_err(|e| NetError::Bind(e.to_string()))?
220	                .bind_addr("[::1]:0")
221	                .map_err(|e| NetError::Bind(e.to_string()))?
222	                .portmapper_config(PortmapperConfig::Disabled),
223	            // All-interfaces like the production scope, but on a caller-fixed
224	            // port so the node is dialable by a static ip:port ticket.
225	            BindScope::Port(p) => builder
226	                .clear_ip_transports()
227	                .bind_addr(std::net::SocketAddr::from((
228	                    std::net::Ipv4Addr::UNSPECIFIED,
229	                    p,
230	                )))
231	                .map_err(|e| NetError::Bind(e.to_string()))?
232	                .bind_addr(std::net::SocketAddr::from((
233	                    std::net::Ipv6Addr::UNSPECIFIED,
234	                    p,
235	                )))
236	                .map_err(|e| NetError::Bind(e.to_string()))?,
237	        };
238	        // The N0 preset already wires the n0 relay default; only override it when the
239	        // policy asks for self-host or disabled.
240	        let builder = match relay {
241	            RelayPolicy::N0Default => builder,
242	            RelayPolicy::SelfHosted(map) => builder.relay_mode(RelayMode::Custom(map)),
243	            RelayPolicy::Disabled => builder.relay_mode(RelayMode::Disabled),
244	        };
245	        // mDNS rides alongside the preset's n0 DNS discovery as an extra address
246	        // lookup, giving LAN-local advertise + resolve-by-id.
247	        let builder = match discovery {
248	            LocalDiscovery::Mdns => {
249	                builder.address_lookup(MdnsAddressLookup::builder().advertise(true))
250	            }
251	            LocalDiscovery::Off => builder,
252	        };
253	        let inner = builder
254	            .bind()
255	            .await
256	            .map_err(|e| NetError::Bind(e.to_string()))?;
257	        Ok(NetEndpoint { inner })
258	    }
259
260	    /// A cheap handle clone (iroh endpoints are internally reference-counted):
261	    /// the SAME bound endpoint — same sockets, same id — usable from
262	    /// concurrent tasks (the meet-probe race spawns one per window token).
263	    pub fn clone_handle(&self) -> NetEndpoint {
264	        NetEndpoint {
265	            inner: self.inner.clone(),
266	        }
267	    }
268
269	    /// This node's public key — its stable subnet name, derived from the live
270	    /// endpoint id. Always a valid ed25519 point (iroh guarantees it).
271	    // [impl->REQ-NET-1]
272	    pub fn node_id(&self) -> PublicKey {
273	        PublicKey::from_bytes(self.inner.id().as_bytes())
274	            .expect("an iroh EndpointId is always a valid ed25519 public key")
275	    }
276
277	    /// The node's stable name as lowercase hex (the `CONTEXT.md` SptNode string).
278	    pub fn node_id_hex(&self) -> String {
279	        self.node_id().to_hex()
280	    }
281
282	    /// This endpoint's dialable address (id + currently-known direct/relay paths).
283	    /// Serialise it (serde) to hand a peer a connect ticket.
284	    pub fn addr(&self) -> EndpointAddr {
285	        self.inner.addr()
286	    }
287
288	    /// Wait until the endpoint has registered with its relays — needed before the
289	    /// [`addr`](Self::addr) ticket is dialable from off-LAN. Not required for a
290	    /// direct/loopback dial where the peer already has a direct address.
291	    pub async fn online(&self) {
292	        self.inner.online().await;
293	    }
294
295	    /// Accept the next inbound connection (already ALPN-filtered to [`SPT_NET_ALPN`]).
296	    /// `None` once the endpoint is closed.
297	    pub async fn accept(&self) -> Option<Incoming> {
298	        self.inner.accept().await
299	    }
300
301	    /// Gracefully close the endpoint: every held connection sends its QUIC
302	    /// CLOSE frame so peers observe the disconnect promptly (a bare drop may
303	    /// tear the runtime down before the close ever flushes, leaving peers to
304	    /// an idle timeout). The D4c presence seam relies on this for prompt
305	    /// disconnected events.
306	    // [impl->REQ-NET-1]
307	    pub async fn close(&self) {
308	        self.inner.close().await;
309	    }
310
311	    /// Dial a peer by address over [`SPT_NET_ALPN`].
312	    // [impl->REQ-NET-1]
313	    pub async fn connect(&self, addr: EndpointAddr) -> Result<Connection, NetError> {
314	        self.connect_with_alpn(addr, SPT_NET_ALPN).await
315	    }
316
317	    /// Dial a peer over an explicit ALPN (the pairing driver dials `SPT_PAIR_ALPN`).
318	    // [impl->REQ-NET-1]
319	    pub async fn connect_with_alpn(
320	        &self,
321	        addr: EndpointAddr,
322	        alpn: &[u8],
323	    ) -> Result<Connection, NetError> {
324	        self.inner
325	            .connect(addr, alpn)
326	            .await
327	            .map_err(|e| NetError::Connect(e.to_string()))
328	    }
329
330	    /// Dial a peer by its node id (public key) alone, letting discovery (mDNS on the
331	    /// LAN, n0 DNS on the WAN) resolve the address. This is the "no ticket needed"
332	    /// path: with [`LocalDiscovery::Mdns`] enabled, two machines on one LAN connect
333	    /// knowing only each other's stable name.
334	    // [impl->REQ-NET-1]
335	    pub async fn connect_by_id(&self, peer: PublicKey) -> Result<Connection, NetError> {
336	        let id = EndpointId::from_bytes(&peer.to_bytes())
337	            .map_err(|e| NetError::Connect(format!("invalid peer id: {e}")))?;
338	        self.connect(EndpointAddr::new(id)).await
339	    }
340
341	    /// Borrow the underlying iroh endpoint — the seam the D4 broker uses to own the
342	    /// endpoint + its live QUIC streams across brain restarts (ADR-0004 §B).
343	    pub fn iroh(&self) -> &Endpoint {
344	        &self.inner
345	    }
346	}
347
348	/// An id-only dialable [`EndpointAddr`] for a node named by its lowercase-hex
349	/// pubkey (the trust store's identity column), letting discovery (mDNS on the
350	/// LAN, n0 DNS on the WAN) resolve the transport address. The peer pump's
351	/// production resolver (D9-1): dial trusted peers knowing only their stable
352	/// name. `None` on a malformed hex — a corrupt trust row must not panic the
353	/// pump.
354	// [impl->REQ-NET-1]
355	pub fn addr_for_node_hex(hex: &str) -> Option<EndpointAddr> {
356	    let pk = spt_proto::identity::PublicKey::from_hex(hex).ok()?;
357	    let id = EndpointId::from_bytes(&pk.to_bytes()).ok()?;
358	    Some(EndpointAddr::new(id))
359	}
360
361	/// The peer's dialable address as OBSERVED from a live connection: the
362	/// handshake-proven remote id + the transport addresses of the connection's
363	/// current (non-closed) network paths. This is what the address-seeding
364	/// writers persist (M8 decisions 14/20, REQ-CONV-1): an address that just
365	/// carried a real connection is the best possible cold-start seed — unlike
366	/// the id-only `addr_for_node_hex`, it dials without a discovery round-trip.
367	/// May be path-less (id-only) when no path is readable; callers skip
368	/// persisting those (they would add nothing over id-only discovery).
369	// [impl->REQ-CONV-1]
370	pub fn conn_remote_addr(conn: &Connection) -> EndpointAddr {
371	    let addrs: Vec<_> = conn
372	        .paths()
373	        .into_iter()
374	        .filter(|p| !p.is_closed())
375	        .map(|p| p.remote_addr().clone())
376	        .collect();
377	    EndpointAddr::from_parts(conn.remote_id(), addrs)
378	}
379
380	#[cfg(test)]
381	mod tests {
382	    use super::*;
383
384	    /// Bind a fully hermetic endpoint: relays disabled, no discovery, loopback
385	    /// sockets only (no real interface touched, no Windows firewall prompt).
386	    async fn bind_hermetic(id: &Identity) -> NetEndpoint {
387	        NetEndpoint::bind_scoped(
388	            id,
389	            RelayPolicy::Disabled,
390	            LocalDiscovery::Off,
391	            vec![SPT_NET_ALPN.to_vec()],
392	            BindScope::Loopback,
393	        )
394	        .await
395	        .expect("hermetic bind")
396	    }
397
398	    /// Accept exactly one connection on `server` and echo a single bidi request as
399	    /// `echo:<req>`. Shared by the loopback + mDNS tests.
400	    fn spawn_echo_once(server: NetEndpoint) -> tokio::task::JoinHandle<()> {
401	        tokio::spawn(async move {
402	            let incoming = server.accept().await.expect("incoming");
403	            let conn = incoming
404	                .accept()
405	                .expect("accept")
406	                .await
407	                .expect("connecting");
408	            let (mut send, mut recv) = conn.accept_bi().await.expect("accept_bi");
409	            let req = recv.read_to_end(1024).await.expect("read req");
410	            let mut echo = b"echo:".to_vec();
411	            echo.extend_from_slice(&req);
412	            send.write_all(&echo).await.expect("write echo");
413	            send.finish().expect("finish");
414	            conn.closed().await;
415	        })
416	    }
417
418	    /// Drive one ping→`echo:ping` round-trip over an established connection.
419	    async fn ping_echo(conn: &Connection) {
420	        let (mut send, mut recv) = conn.open_bi().await.expect("open_bi");
421	        send.write_all(b"ping").await.expect("write ping");
422	        send.finish().expect("finish");
423	        let resp = recv.read_to_end(1024).await.expect("read resp");
424	        assert_eq!(resp, b"echo:ping", "bidi echo over QUIC");
425	    }
426
427	    // [unit->REQ-MESH-1] the keep-warm invariant: keep-alive must be strictly
428	    // less than the idle timeout, or a PING never lands before the idle timer
429	    // fires and a quiet member connection drops (forcing needless re-proof).
430	    // Asserted at the constant seam (iroh's QuicTransportConfig has no getters);
431	    // the production path feeds these same constants into spt_transport_config.
432	    #[test]
433	    fn keep_alive_is_under_idle_timeout() {
434	        assert!(
435	            MESH_KEEP_ALIVE < MESH_MAX_IDLE,
436	            "keep-alive {MESH_KEEP_ALIVE:?} must be < idle {MESH_MAX_IDLE:?}"
437	        );
438	        // Constructs without panicking (the IdleTimeout conversion is valid).
439	        let _ = spt_transport_config();
440	    }
441
442	    // [unit->REQ-NET-1] the endpoint binds to the node's own ed25519 identity: the
443	    // iroh EndpointId equals the node's public key (same seed → same keypair).
444	    #[tokio::test]
445	    async fn endpoint_id_binds_to_identity() {
446	        let id = Identity::generate();
447	        let ep = bind_hermetic(&id).await;
448	        assert_eq!(
449	            ep.node_id(),
450	            id.public_key(),
451	            "endpoint id must equal node pubkey"
452	        );
453	        assert_eq!(ep.node_id_hex(), id.public_key().to_hex());
454	    }
455
456	    // [unit->REQ-NET-1] two endpoints connect + exchange a bidi message over loopback
457	    // QUIC with relays DISABLED, no discovery, and loopback-only sockets — fully
458	    // hermetic (no network), proving the ALPN + connect/accept path by direct
459	    // address. (The cross-node WAN `int` stage is the two-host E2E at D9; this is
460	    // the component-level evidence.)
461	    #[tokio::test]
462	    async fn two_endpoints_loopback_echo() {
463	        let server = bind_hermetic(&Identity::generate()).await;
464	        let server_addr = server.addr();
465	        let srv = spawn_echo_once(server);
466
467	        let client = bind_hermetic(&Identity::generate()).await;
468	        let conn = client.connect(server_addr).await.expect("connect");
469	        ping_echo(&conn).await;
470	        conn.close(0u32.into(), b"done");
471
472	        srv.await.expect("server task joins");
473	    }
474
475	    // [unit->REQ-NET-1] mDNS LAN discovery: two endpoints, each with mDNS on, connect
476	    // knowing only the peer's node id (no address ticket) — the address is resolved
477	    // over mDNS. #[ignore]'d because it needs working LAN multicast (swarm-discovery),
478	    // which CI sandboxes don't reliably provide; run locally with `--ignored` or on
479	    // the two-host rig at D9. Validated locally 2026-06-03 (single-host multicast).
480	    #[tokio::test]
481	    #[ignore = "needs LAN multicast; run with --ignored or on the D9 two-host rig"]
482	    async fn mdns_discovery_connects_by_id() {
483	        let server = NetEndpoint::bind(
484	            &Identity::generate(),
485	            RelayPolicy::Disabled,
486	            LocalDiscovery::Mdns,
487	        )
488	        .await
489	        .expect("server bind");
490	        let server_pk = server.node_id();
491	        let srv = spawn_echo_once(server);
492
493	        let client = NetEndpoint::bind(
494	            &Identity::generate(),
495	            RelayPolicy::Disabled,
496	            LocalDiscovery::Mdns,
497	        )
498	        .await
499	        .expect("client bind");
500	        // Give mDNS a moment to advertise + resolve before dialing by id alone.
501	        tokio::time::sleep(std::time::Duration::from_secs(3)).await;
502	        let conn = client
503	            .connect_by_id(server_pk)
504	            .await
505	            .expect("connect by id resolves over mDNS");
506	        ping_echo(&conn).await;
507	        conn.close(0u32.into(), b"done");
508
509	        srv.await.expect("server task joins");
510	    }
511	}
512

//! The Iroh endpoint — the node's WAN/LAN presence (D1).
//!
//! [`NetEndpoint`] wraps an `iroh::Endpoint` bound to the node's **own** Ed25519
//! identity (`spt-proto`'s [`Identity`]): the iroh `EndpointId` *is* the node's
//! public key (same 32-byte seed → same ed25519 keypair), so a node's network
//! address and its stable name (`CONTEXT.md` "SptNode") are one and the same — no
//! second identity to reconcile. Connections speak the [`SPT_NET_ALPN`] protocol.
//!
//! Relay policy ([`RelayPolicy`]) is the ADR-0002 v1 relay stance made concrete:
//! n0's public relays by default for zero-config onboarding, a self-host escape
//! hatch, and a disabled mode (LAN/air-gapped + hermetic tests). See
//! [`RELAY_DISCLOSURE`] for the plain-language disclosure that ships to users.
//!
//! API shape is pinned to iroh 0.98.2 (Spike #2 — the API churns every release).
//! This crate owns the *mechanism*; the broker (ADR-0004 §B, D4) owns the *process*
//! that holds a live endpoint across brain restarts — [`NetEndpoint::iroh`] is the
//! escape hatch the daemon uses to take ownership.

use std::time::Duration;

use iroh::address_lookup::MdnsAddressLookup;
use iroh::endpoint::{presets, IdleTimeout, Incoming, PortmapperConfig, QuicTransportConfig};
use iroh::{Endpoint, EndpointId, RelayMap, RelayMode, SecretKey};
use spt_proto::identity::{Identity, PublicKey};

// Re-exported for the daemon (D4): the broker holds live QUIC connections +
// streams in its tables and dialable addresses in its IPC frames without growing
// its own iroh dependency edge — the exact version pin (0.98.2) stays in this
// one crate.
pub use iroh::endpoint::{Connection, RecvStream, SendStream};
pub use iroh::EndpointAddr;

/// The ALPN every spt-core node-to-node QUIC connection negotiates. Versioned so a
/// future wire break is a new protocol id, not a silent incompatibility.
pub const SPT_NET_ALPN: &[u8] = b"spt-core/net/0";

/// QUIC keep-alive PING interval — the mesh's "stay warm" knob (REQ-MESH-1).
/// A member connection that proves seed-knowledge once at connect must then stay
/// open through indefinite idleness, so re-proof is a restart/partition/rotation
/// event, never a per-message tax. iroh's default leaves keep-alive **off** and
/// idles a quiet connection out at 30s; without this a meshed-but-quiet member
/// would silently drop and re-handshake constantly. Must be strictly less than
/// [`MESH_MAX_IDLE`] so a PING always lands before the idle timer fires.
const MESH_KEEP_ALIVE: Duration = Duration::from_secs(15);

/// QUIC max idle timeout for spt connections. A live PING every
/// [`MESH_KEEP_ALIVE`] resets this timer, so a connection with both ends up
/// stays open indefinitely; a real partition or peer death tears it down within
/// this bound (then the next dial re-proves membership). Bounded, not infinite:
/// a dead peer must eventually leave the broker conn table (the D4c presence seam).
const MESH_MAX_IDLE: Duration = Duration::from_secs(60);

/// The spt QUIC transport config: keep-alive on, bounded idle timeout (see
/// [`MESH_KEEP_ALIVE`] / [`MESH_MAX_IDLE`]). Factored out so the keep-warm
/// invariant (`keep_alive < max_idle`) is assertable without reading back iroh's
/// opaque config type.
// [impl->REQ-MESH-1]
fn spt_transport_config() -> QuicTransportConfig {
    QuicTransportConfig::builder()
        .keep_alive_interval(MESH_KEEP_ALIVE)
        .max_idle_timeout(Some(
            IdleTimeout::try_from(MESH_MAX_IDLE).expect("60s is a valid QUIC idle timeout"),
        ))
        .build()
}

/// Plain-language disclosure of the soft runtime dependency on n0-operated relay
/// infrastructure (ADR-0002 v1 relay stance). Surfaced to the user by the CLI/setup
/// path; the dependency is operational + metadata-only (traffic is end-to-end
/// encrypted) and escapable via [`RelayPolicy::SelfHosted`].
// [impl->REQ-NET-2]
pub const RELAY_DISCLOSURE: &str = "\
By default spt connects through n0's free public relay servers to find your other \
machines and to assist NAT hole-punching. Relays only help peers locate and connect \
to each other — your traffic between machines is end-to-end encrypted and the relay \
cannot read it. You can run your own relay instead (self-host) or disable relays \
entirely for a LAN-only/air-gapped setup.";

/// Where this endpoint looks for relay assistance (ADR-0002). Maps onto iroh's
/// [`RelayMode`]; the default ([`RelayPolicy::N0Default`]) is the zero-config path.
// [impl->REQ-NET-2]
#[derive(Debug, Clone)]
pub enum RelayPolicy {
    /// n0's public production relays — the zero-config "just works" default.
    N0Default,
    /// Self-hosted relay(s) — the principled escape hatch from the n0 dependency.
    SelfHosted(RelayMap),
    /// No relays: direct/LAN connectivity only. Used by air-gapped LANs and the
    /// hermetic loopback tests (no network reached at all).
    Disabled,
}

/// Whether the endpoint also runs **mDNS** LAN discovery on top of the preset's n0
/// DNS discovery. With mDNS on, a node advertises itself and resolves peers by id
/// alone on the local network, so two machines on the same LAN find each other with
/// no ticket exchange and no relay round-trip ([`NetEndpoint::connect_by_id`]).
// [impl->REQ-NET-1]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum LocalDiscovery {
    /// Advertise + resolve over mDNS (swarm-discovery) in addition to n0 DNS.
    Mdns,
    /// No mDNS — WAN discovery only (and direct addresses). Used by hermetic tests.
    Off,
}

/// Which interfaces the endpoint's UDP sockets bind — a mechanical knob, not
/// policy. [`BindScope::Loopback`] exists for hermetic tests: a loopback-only
/// listener never trips the Windows Defender Firewall "allow access?" prompt
/// (the prompt fires per freshly-built test executable that listens on
/// `0.0.0.0`), and it makes "no network reached" literally true at the socket.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum BindScope {
    /// Bind the unspecified addresses (`0.0.0.0` / `[::]`) — all interfaces.
    /// The production scope; required for any off-host connectivity.
    All,
    /// Bind `127.0.0.1` / `[::1]` only. Hermetic tests: loopback dials work,
    /// nothing can reach (or prompt about) a real interface.
    Loopback,
    /// Bind all interfaces on an **explicit UDP port** (`0.0.0.0:p` / `[::]:p`)
    /// instead of an ephemeral one. A node whose port is stable can be named by
    /// a static `ip:port` ticket (firewall allow-rules, the two-host rig's
    /// env-derived peer addresses) with no relay/discovery infrastructure.
    Port(u16),
}

/// Failures binding or dialing the endpoint. Wraps iroh's error chain as a message
/// so iroh's churning error types don't leak into this crate's public surface
/// (matches the workspace's hand-rolled, dependency-free error convention).
#[derive(Debug)]
pub enum NetError {
    /// The endpoint could not bind its UDP socket / initialise the transport.
    Bind(String),
    /// Dialing a peer failed (no path, ALPN mismatch, peer down, …).
    Connect(String),
}

impl std::fmt::Display for NetError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            NetError::Bind(e) => write!(f, "endpoint bind failed: {e}"),
            NetError::Connect(e) => write!(f, "connect failed: {e}"),
        }
    }
}

impl std::error::Error for NetError {}

/// A bound spt-core network endpoint: the node's reachable presence on the subnet.
pub struct NetEndpoint {
    inner: Endpoint,
}

impl NetEndpoint {
    /// Bind an endpoint to this node's identity under the given relay policy.
    ///
    /// The iroh secret key is derived from the identity's 32-byte ed25519 seed, so
    /// the resulting `EndpointId` equals the node's [`PublicKey`] — the address and
    /// the name are the same value.
    // [impl->REQ-NET-1] [impl->REQ-NET-2]
    pub async fn bind(
        identity: &Identity,
        relay: RelayPolicy,
        discovery: LocalDiscovery,
    ) -> Result<NetEndpoint, NetError> {
        Self::bind_with_alpns(identity, relay, discovery, vec![SPT_NET_ALPN.to_vec()]).await
    }

    /// Bind on an explicit ALPN set instead of the default trusted [`SPT_NET_ALPN`].
    ///
    /// The pairing driver uses this to stand up a **dedicated pre-trust endpoint** bound
    /// only to the pairing ALPN (`super::pairing::wire::SPT_PAIR_ALPN`) — the pre-trust
    /// accept loop must not share the trusted message surface (ADR-0005; `pairing.rs`
    /// "Why a dedicated pre-trust ALPN"). All relay/discovery handling is identical to
    /// [`bind`](Self::bind).
    // [impl->REQ-NET-1] [impl->REQ-NET-2]
    pub async fn bind_with_alpns(
        identity: &Identity,
        relay: RelayPolicy,
        discovery: LocalDiscovery,
        alpns: Vec<Vec<u8>>,
    ) -> Result<NetEndpoint, NetError> {
        Self::bind_scoped(identity, relay, discovery, alpns, BindScope::All).await
    }

    /// The full bind form: [`bind_with_alpns`](Self::bind_with_alpns) plus an
    /// explicit [`BindScope`]. Hermetic tests pass [`BindScope::Loopback`] so the
    /// endpoint never listens on a real interface (and never triggers a Windows
    /// firewall prompt per test binary); everything else delegates here with
    /// [`BindScope::All`].
    // [impl->REQ-NET-1] [impl->REQ-NET-2]
    pub async fn bind_scoped(
        identity: &Identity,
        relay: RelayPolicy,
        discovery: LocalDiscovery,
        alpns: Vec<Vec<u8>>,
        scope: BindScope,
    ) -> Result<NetEndpoint, NetError> {
        // iroh and spt-proto both build the ed25519 keypair from the same 32-byte
        // seed, so this preserves the node's stable public-key name on the wire.
        let secret = SecretKey::from_bytes(&identity.seed());
        let builder = Endpoint::builder(presets::N0)
            .secret_key(secret)
            .alpns(alpns)
            // Keep meshed member connections warm (REQ-MESH-1): seed-proof runs
            // once at connect, so a quiet connection must not idle out and force
            // a re-handshake. Transport-only, orthogonal to BindScope.
            .transport_config(spt_transport_config());
        // Loopback scope replaces the builder's pre-configured unspecified binds
        // (0.0.0.0 / [::]) with loopback-only sockets, AND disables iroh's
        // portmapper (UPnP/PCP/NAT-PMP — on by default), which would otherwise
        // bind real-interface UDP sockets for its SSDP/NAT-PMP probes: useless
        // on loopback, and the other source of per-test-binary Windows firewall
        // prompts.
        let builder = match scope {
            BindScope::All => builder,
            BindScope::Loopback => builder
                .clear_ip_transports()
                .bind_addr("127.0.0.1:0")
                .map_err(|e| NetError::Bind(e.to_string()))?
                .bind_addr("[::1]:0")
                .map_err(|e| NetError::Bind(e.to_string()))?
                .portmapper_config(PortmapperConfig::Disabled),
            // All-interfaces like the production scope, but on a caller-fixed
            // port so the node is dialable by a static ip:port ticket.
            BindScope::Port(p) => builder
                .clear_ip_transports()
                .bind_addr(std::net::SocketAddr::from((
                    std::net::Ipv4Addr::UNSPECIFIED,
                    p,
                )))
                .map_err(|e| NetError::Bind(e.to_string()))?
                .bind_addr(std::net::SocketAddr::from((
                    std::net::Ipv6Addr::UNSPECIFIED,
                    p,
                )))
                .map_err(|e| NetError::Bind(e.to_string()))?,
        };
        // The N0 preset already wires the n0 relay default; only override it when the
        // policy asks for self-host or disabled.
        let builder = match relay {
            RelayPolicy::N0Default => builder,
            RelayPolicy::SelfHosted(map) => builder.relay_mode(RelayMode::Custom(map)),
            RelayPolicy::Disabled => builder.relay_mode(RelayMode::Disabled),
        };
        // mDNS rides alongside the preset's n0 DNS discovery as an extra address
        // lookup, giving LAN-local advertise + resolve-by-id.
        let builder = match discovery {
            LocalDiscovery::Mdns => {
                builder.address_lookup(MdnsAddressLookup::builder().advertise(true))
            }
            LocalDiscovery::Off => builder,
        };
        let inner = builder
            .bind()
            .await
            .map_err(|e| NetError::Bind(e.to_string()))?;
        Ok(NetEndpoint { inner })
    }

    /// A cheap handle clone (iroh endpoints are internally reference-counted):
    /// the SAME bound endpoint — same sockets, same id — usable from
    /// concurrent tasks (the meet-probe race spawns one per window token).
    pub fn clone_handle(&self) -> NetEndpoint {
        NetEndpoint {
            inner: self.inner.clone(),
        }
    }

    /// This node's public key — its stable subnet name, derived from the live
    /// endpoint id. Always a valid ed25519 point (iroh guarantees it).
    // [impl->REQ-NET-1]
    pub fn node_id(&self) -> PublicKey {
        PublicKey::from_bytes(self.inner.id().as_bytes())
            .expect("an iroh EndpointId is always a valid ed25519 public key")
    }

    /// The node's stable name as lowercase hex (the `CONTEXT.md` SptNode string).
    pub fn node_id_hex(&self) -> String {
        self.node_id().to_hex()
    }

    /// This endpoint's dialable address (id + currently-known direct/relay paths).
    /// Serialise it (serde) to hand a peer a connect ticket.
    pub fn addr(&self) -> EndpointAddr {
        self.inner.addr()
    }

    /// Wait until the endpoint has registered with its relays — needed before the
    /// [`addr`](Self::addr) ticket is dialable from off-LAN. Not required for a
    /// direct/loopback dial where the peer already has a direct address.
    pub async fn online(&self) {
        self.inner.online().await;
    }

    /// Accept the next inbound connection (already ALPN-filtered to [`SPT_NET_ALPN`]).
    /// `None` once the endpoint is closed.
    pub async fn accept(&self) -> Option<Incoming> {
        self.inner.accept().await
    }

    /// Gracefully close the endpoint: every held connection sends its QUIC
    /// CLOSE frame so peers observe the disconnect promptly (a bare drop may
    /// tear the runtime down before the close ever flushes, leaving peers to
    /// an idle timeout). The D4c presence seam relies on this for prompt
    /// disconnected events.
    // [impl->REQ-NET-1]
    pub async fn close(&self) {
        self.inner.close().await;
    }

    /// Dial a peer by address over [`SPT_NET_ALPN`].
    // [impl->REQ-NET-1]
    pub async fn connect(&self, addr: EndpointAddr) -> Result<Connection, NetError> {
        self.connect_with_alpn(addr, SPT_NET_ALPN).await
    }

    /// Dial a peer over an explicit ALPN (the pairing driver dials `SPT_PAIR_ALPN`).
    // [impl->REQ-NET-1]
    pub async fn connect_with_alpn(
        &self,
        addr: EndpointAddr,
        alpn: &[u8],
    ) -> Result<Connection, NetError> {
        self.inner
            .connect(addr, alpn)
            .await
            .map_err(|e| NetError::Connect(e.to_string()))
    }

    /// Dial a peer by its node id (public key) alone, letting discovery (mDNS on the
    /// LAN, n0 DNS on the WAN) resolve the address. This is the "no ticket needed"
    /// path: with [`LocalDiscovery::Mdns`] enabled, two machines on one LAN connect
    /// knowing only each other's stable name.
    // [impl->REQ-NET-1]
    pub async fn connect_by_id(&self, peer: PublicKey) -> Result<Connection, NetError> {
        let id = EndpointId::from_bytes(&peer.to_bytes())
            .map_err(|e| NetError::Connect(format!("invalid peer id: {e}")))?;
        self.connect(EndpointAddr::new(id)).await
    }

    /// Borrow the underlying iroh endpoint — the seam the D4 broker uses to own the
    /// endpoint + its live QUIC streams across brain restarts (ADR-0004 §B).
    pub fn iroh(&self) -> &Endpoint {
        &self.inner
    }
}

/// An id-only dialable [`EndpointAddr`] for a node named by its lowercase-hex
/// pubkey (the trust store's identity column), letting discovery (mDNS on the
/// LAN, n0 DNS on the WAN) resolve the transport address. The peer pump's
/// production resolver (D9-1): dial trusted peers knowing only their stable
/// name. `None` on a malformed hex — a corrupt trust row must not panic the
/// pump.
// [impl->REQ-NET-1]
pub fn addr_for_node_hex(hex: &str) -> Option<EndpointAddr> {
    let pk = spt_proto::identity::PublicKey::from_hex(hex).ok()?;
    let id = EndpointId::from_bytes(&pk.to_bytes()).ok()?;
    Some(EndpointAddr::new(id))
}

/// The peer's dialable address as OBSERVED from a live connection: the
/// handshake-proven remote id + the transport addresses of the connection's
/// current (non-closed) network paths. This is what the address-seeding
/// writers persist (M8 decisions 14/20, REQ-CONV-1): an address that just
/// carried a real connection is the best possible cold-start seed — unlike
/// the id-only `addr_for_node_hex`, it dials without a discovery round-trip.
/// May be path-less (id-only) when no path is readable; callers skip
/// persisting those (they would add nothing over id-only discovery).
// [impl->REQ-CONV-1]
pub fn conn_remote_addr(conn: &Connection) -> EndpointAddr {
    let addrs: Vec<_> = conn
        .paths()
        .into_iter()
        .filter(|p| !p.is_closed())
        .map(|p| p.remote_addr().clone())
        .collect();
    EndpointAddr::from_parts(conn.remote_id(), addrs)
}

#[cfg(test)]
mod tests {
    use super::*;

    /// Bind a fully hermetic endpoint: relays disabled, no discovery, loopback
    /// sockets only (no real interface touched, no Windows firewall prompt).
    async fn bind_hermetic(id: &Identity) -> NetEndpoint {
        NetEndpoint::bind_scoped(
            id,
            RelayPolicy::Disabled,
            LocalDiscovery::Off,
            vec![SPT_NET_ALPN.to_vec()],
            BindScope::Loopback,
        )
        .await
        .expect("hermetic bind")
    }

    /// Accept exactly one connection on `server` and echo a single bidi request as
    /// `echo:<req>`. Shared by the loopback + mDNS tests.
    fn spawn_echo_once(server: NetEndpoint) -> tokio::task::JoinHandle<()> {
        tokio::spawn(async move {
            let incoming = server.accept().await.expect("incoming");
            let conn = incoming
                .accept()
                .expect("accept")
                .await
                .expect("connecting");
            let (mut send, mut recv) = conn.accept_bi().await.expect("accept_bi");
            let req = recv.read_to_end(1024).await.expect("read req");
            let mut echo = b"echo:".to_vec();
            echo.extend_from_slice(&req);
            send.write_all(&echo).await.expect("write echo");
            send.finish().expect("finish");
            conn.closed().await;
        })
    }

    /// Drive one ping→`echo:ping` round-trip over an established connection.
    async fn ping_echo(conn: &Connection) {
        let (mut send, mut recv) = conn.open_bi().await.expect("open_bi");
        send.write_all(b"ping").await.expect("write ping");
        send.finish().expect("finish");
        let resp = recv.read_to_end(1024).await.expect("read resp");
        assert_eq!(resp, b"echo:ping", "bidi echo over QUIC");
    }

    // [unit->REQ-MESH-1] the keep-warm invariant: keep-alive must be strictly
    // less than the idle timeout, or a PING never lands before the idle timer
    // fires and a quiet member connection drops (forcing needless re-proof).
    // Asserted at the constant seam (iroh's QuicTransportConfig has no getters);
    // the production path feeds these same constants into spt_transport_config.
    #[test]
    fn keep_alive_is_under_idle_timeout() {
        assert!(
            MESH_KEEP_ALIVE < MESH_MAX_IDLE,
            "keep-alive {MESH_KEEP_ALIVE:?} must be < idle {MESH_MAX_IDLE:?}"
        );
        // Constructs without panicking (the IdleTimeout conversion is valid).
        let _ = spt_transport_config();
    }

    // [unit->REQ-NET-1] the endpoint binds to the node's own ed25519 identity: the
    // iroh EndpointId equals the node's public key (same seed → same keypair).
    #[tokio::test]
    async fn endpoint_id_binds_to_identity() {
        let id = Identity::generate();
        let ep = bind_hermetic(&id).await;
        assert_eq!(
            ep.node_id(),
            id.public_key(),
            "endpoint id must equal node pubkey"
        );
        assert_eq!(ep.node_id_hex(), id.public_key().to_hex());
    }

    // [unit->REQ-NET-1] two endpoints connect + exchange a bidi message over loopback
    // QUIC with relays DISABLED, no discovery, and loopback-only sockets — fully
    // hermetic (no network), proving the ALPN + connect/accept path by direct
    // address. (The cross-node WAN `int` stage is the two-host E2E at D9; this is
    // the component-level evidence.)
    #[tokio::test]
    async fn two_endpoints_loopback_echo() {
        let server = bind_hermetic(&Identity::generate()).await;
        let server_addr = server.addr();
        let srv = spawn_echo_once(server);

        let client = bind_hermetic(&Identity::generate()).await;
        let conn = client.connect(server_addr).await.expect("connect");
        ping_echo(&conn).await;
        conn.close(0u32.into(), b"done");

        srv.await.expect("server task joins");
    }

    // [unit->REQ-NET-1] mDNS LAN discovery: two endpoints, each with mDNS on, connect
    // knowing only the peer's node id (no address ticket) — the address is resolved
    // over mDNS. #[ignore]'d because it needs working LAN multicast (swarm-discovery),
    // which CI sandboxes don't reliably provide; run locally with `--ignored` or on
    // the two-host rig at D9. Validated locally 2026-06-03 (single-host multicast).
    #[tokio::test]
    #[ignore = "needs LAN multicast; run with --ignored or on the D9 two-host rig"]
    async fn mdns_discovery_connects_by_id() {
        let server = NetEndpoint::bind(
            &Identity::generate(),
            RelayPolicy::Disabled,
            LocalDiscovery::Mdns,
        )
        .await
        .expect("server bind");
        let server_pk = server.node_id();
        let srv = spawn_echo_once(server);

        let client = NetEndpoint::bind(
            &Identity::generate(),
            RelayPolicy::Disabled,
            LocalDiscovery::Mdns,
        )
        .await
        .expect("client bind");
        // Give mDNS a moment to advertise + resolve before dialing by id alone.
        tokio::time::sleep(std::time::Duration::from_secs(3)).await;
        let conn = client
            .connect_by_id(server_pk)
            .await
            .expect("connect by id resolves over mDNS");
        ping_echo(&conn).await;
        conn.close(0u32.into(), b"done");

        srv.await.expect("server task joins");
    }
}


1	# Robust WAN subnet join — design
2
3	> Source: `/grill-with-docs` session 2026-06-27 (operator + doyle), off a confirmed field incident
4	> (SCELTOUIN could not join SPT_DEV over WAN — half-broken IPv6 silently burned the rendezvous
5	> window). Decision record: **ADR-0030**. Glossary correction landed in CONTEXT.md ("link discovery").
6	> Each item below lands as a tagged `REQ-*` in `traceable-reqs.toml`; this is the next milestone's spec.
7
8	## Background (the confirmed root cause)
9
10	The joiner dials an **ephemeral, 30 s-rotating** rendezvous id derived from the **public**
11	`(subnet-name, TOTP-epoch)` — *not* the secret code. iroh resolves that id via n0 DNS
12	(`dns.iroh.link`) + relay on WAN, mDNS on LAN, then the responder answers with its real address and
13	the SPAKE2 ceremony runs (the code is the PAKE password). SCELTOUIN had **half-broken IPv6** (AAAA
14	resolves, path dead); iroh's dual-stack attempts to dead IPv6 candidates silently timed out, blowing
15	the ±1 window (~90 s of token acceptance; `SEARCH_DEADLINE` 75 s). Symptoms: usually a silent
16	timeout; occasionally a slow meet whose pre-entered code had rotated → "wrong code". Refuted cheaply
17	first: clock/NTP (offset +0.09 s), Windows firewall (inbound-UDP allow present), n0 relay reach (fine).
18	The discriminator: `Test-NetConnection dns.iroh.link -Port 443` — IPv6 addr **fails**, IPv4 succeeds.
19
20	## A. Meet-before-code — two-phase join → `REQ-JOIN-TWO-PHASE`
21
22	The meet selector is public `(name, epoch)`; the code is only the ceremony password. So collect the
23	code **after** the meet and use it immediately — fresh at ceremony regardless of discovery time.
24
25	- **Protocol (one stream, extend `brain.pair_join`, `brain.rs:1009`):**
26	  `PairMeetReq{subnet}` → daemon runs the meet `(name, current-epoch)` (`pairhost.rs` `connect_seed_holder`
27	  / `meet.rs` `dial_via_rendezvous`), resolving the seed-holder's **real, stable** pairing address →
28	  `MetMember` event → CLI prompts the code (`cli.rs` `cmd_subnet_join`, `:6236`) → `PairCodeSubmit{code}`
29	  → daemon dials the held real-address on `SPT_PAIR_ALPN` + SPAKE2 → `PairJoined`/`PairFail`.
30	- **Held-address state machine:** the daemon caches the resolved real-address between `MetMember` and
31	  `PairCodeSubmit`, bounded by a **5-minute wait-for-code timeout**. The real-address is the member's
32	  stable endpoint (not the rotating rendezvous id), so it does not age within the hold.
33	- **Wrong-code retry re-runs the ceremony only** against the held address — no re-search (a property
34	  of holding the address). Bounded by the same 5-minute hold.
35	- **`--code` path unchanged (one-shot):** code supplied at invocation → no prompt; meet then ceremony
36	  in sequence. Relies on B (fast discovery) to keep the supplied code in its window; on staleness it
37	  fails loudly per C (no silent burn).
38	- **Security unchanged:** the meet is pre-trust/unauthenticated (`SPT_PAIR_MEET_ALPN`); auth stays in
39	  SPAKE2. Meeting before the code is entered weakens nothing.
40	- Activate impl+unit+int (int = a real two-phase join over a live meet; wrong-code retry hits ceremony
41	  only; the 5-minute hold times out cleanly).
42
43	## B. Per-family bind gate → `REQ-NET-FAMILY-GATE`
44
45	- **At `NetEndpoint::bind` (`endpoint.rs`), probe each IP family's reachability ONCE** and bind only
46	  the families that work: dual-stack when both healthy; IPv4-only when IPv6 is the half-broken
47	  AAAA-resolves-but-dead kind; **IPv6-only when IPv4 is the dead one** (drop the *dead* family — never
48	  a fixed prefer-IPv4; IPv6-only networks must keep working). Re-evaluated on daemon restart.
49	- **Probe shape (impl-verify for the executor):** a short-timeout reachability check per family —
50	  e.g. a bounded connect to the n0 relay's A / AAAA, or a default-route presence check — kept cheap so
51	  it does not slow bind. Confirm the iroh API for binding a single family (omit the v6/v4 socket) and
52	  whether iroh's own hole-punch already fast-fails candidates (may narrow the fix).
53	- **Override knobs:** `SPT_DISABLE_IPV6` / `SPT_DISABLE_IPV4` force a family off (escape hatch +
54	  determinism + testing), mirroring `SPT_NTP_SERVER`. A forced family is never bound regardless of the
55	  health probe.
56	- Reusable beyond join — every spt connection benefits. Activate impl+unit.
57
58	## C. Diagnosable join → `REQ-JOIN-DIAGNOSTICS`
59
60	- **Live progress** during the meet (replace the one-shot "Searching…" at `cli.rs:6268` with periodic
61	  elapsed / deadline output) so silence never reads as a hang.
62	- **Detailed failure** on meet-exhaustion — rendezvous candidates + families attempted (IPv4/IPv6),
63	  relay vs direct, the last concrete error — surfaced **before** any code prompt. (`connect_seed_holder`
64	  `pairhost.rs:437` and `dial_via_rendezvous` `meet.rs:281` currently swallow per-attempt errors —
65	  thread the last error up with attempt context.)
66	- **Propagate the terminal event:** `brain.rs:1024` `_ => continue` must deliver a daemon
67	  `NoSeedHolder`/`PairFail` to the CLI as a printed error, not discard it (this is *why the user saw
68	  nothing*).
69	- **`--verbose` / `SPT_LOG` discovery trace:** per-probe derived id, discovery path taken (mDNS / n0
70	  DNS / relay), per-family timeouts — opt-in, so the next WAN-join issue is self-serve. No knob exists
71	  today.
72	- Activate impl+unit.
73
74	## Cross-cutting
75
76	- **CONTEXT.md** corrected (done this session): **TOTP-epoch** (public, routes the meet) vs
77	  **TOTP-code** (secret, SPAKE2 only); _Avoid_ "the code routes discovery".
78	- **ADR-0030** doc-ratifies the cluster. Traceability: seed all three `REQ-*` with `required_stages =
79	  ["doc"]` (ADR + this doc are the doc evidence); impl/unit/int activate at the build milestone (rule 5).
80	- **Known kin (the other two silent join traps):** `docs/DEFERRED.md` row 23 (Windows-firewall inbound
81	  UDP drop) and row 30 (clock skew → wrong epoch). Both are refuted-but-related; the diagnostics (C)
82	  would have made each self-evident.
83	- **Public docs** (ride the build): MANIFEST/reference unaffected (no manifest surface); gh-pages
84	  `networking/overview` + a join/pairing troubleshooting note (families, `SPT_DISABLE_IPV6`, `--verbose`).
85

# Robust WAN subnet join — design

> Source: `/grill-with-docs` session 2026-06-27 (operator + doyle), off a confirmed field incident
> (SCELTOUIN could not join SPT_DEV over WAN — half-broken IPv6 silently burned the rendezvous
> window). Decision record: **ADR-0030**. Glossary correction landed in CONTEXT.md ("link discovery").
> Each item below lands as a tagged `REQ-*` in `traceable-reqs.toml`; this is the next milestone's spec.

## Background (the confirmed root cause)

The joiner dials an **ephemeral, 30 s-rotating** rendezvous id derived from the **public**
`(subnet-name, TOTP-epoch)` — *not* the secret code. iroh resolves that id via n0 DNS
(`dns.iroh.link`) + relay on WAN, mDNS on LAN, then the responder answers with its real address and
the SPAKE2 ceremony runs (the code is the PAKE password). SCELTOUIN had **half-broken IPv6** (AAAA
resolves, path dead); iroh's dual-stack attempts to dead IPv6 candidates silently timed out, blowing
the ±1 window (~90 s of token acceptance; `SEARCH_DEADLINE` 75 s). Symptoms: usually a silent
timeout; occasionally a slow meet whose pre-entered code had rotated → "wrong code". Refuted cheaply
first: clock/NTP (offset +0.09 s), Windows firewall (inbound-UDP allow present), n0 relay reach (fine).
The discriminator: `Test-NetConnection dns.iroh.link -Port 443` — IPv6 addr **fails**, IPv4 succeeds.

## A. Meet-before-code — two-phase join → `REQ-JOIN-TWO-PHASE`

The meet selector is public `(name, epoch)`; the code is only the ceremony password. So collect the
code **after** the meet and use it immediately — fresh at ceremony regardless of discovery time.

- **Protocol (one stream, extend `brain.pair_join`, `brain.rs:1009`):**
  `PairMeetReq{subnet}` → daemon runs the meet `(name, current-epoch)` (`pairhost.rs` `connect_seed_holder`
  / `meet.rs` `dial_via_rendezvous`), resolving the seed-holder's **real, stable** pairing address →
  `MetMember` event → CLI prompts the code (`cli.rs` `cmd_subnet_join`, `:6236`) → `PairCodeSubmit{code}`
  → daemon dials the held real-address on `SPT_PAIR_ALPN` + SPAKE2 → `PairJoined`/`PairFail`.
- **Held-address state machine:** the daemon caches the resolved real-address between `MetMember` and
  `PairCodeSubmit`, bounded by a **5-minute wait-for-code timeout**. The real-address is the member's
  stable endpoint (not the rotating rendezvous id), so it does not age within the hold.
- **Wrong-code retry re-runs the ceremony only** against the held address — no re-search (a property
  of holding the address). Bounded by the same 5-minute hold.
- **`--code` path unchanged (one-shot):** code supplied at invocation → no prompt; meet then ceremony
  in sequence. Relies on B (fast discovery) to keep the supplied code in its window; on staleness it
  fails loudly per C (no silent burn).
- **Security unchanged:** the meet is pre-trust/unauthenticated (`SPT_PAIR_MEET_ALPN`); auth stays in
  SPAKE2. Meeting before the code is entered weakens nothing.
- Activate impl+unit+int (int = a real two-phase join over a live meet; wrong-code retry hits ceremony
  only; the 5-minute hold times out cleanly).

## B. Per-family bind gate → `REQ-NET-FAMILY-GATE`

- **At `NetEndpoint::bind` (`endpoint.rs`), probe each IP family's reachability ONCE** and bind only
  the families that work: dual-stack when both healthy; IPv4-only when IPv6 is the half-broken
  AAAA-resolves-but-dead kind; **IPv6-only when IPv4 is the dead one** (drop the *dead* family — never
  a fixed prefer-IPv4; IPv6-only networks must keep working). Re-evaluated on daemon restart.
- **Probe shape (impl-verify for the executor):** a short-timeout reachability check per family —
  e.g. a bounded connect to the n0 relay's A / AAAA, or a default-route presence check — kept cheap so
  it does not slow bind. Confirm the iroh API for binding a single family (omit the v6/v4 socket) and
  whether iroh's own hole-punch already fast-fails candidates (may narrow the fix).
- **Override knobs:** `SPT_DISABLE_IPV6` / `SPT_DISABLE_IPV4` force a family off (escape hatch +
  determinism + testing), mirroring `SPT_NTP_SERVER`. A forced family is never bound regardless of the
  health probe.
- Reusable beyond join — every spt connection benefits. Activate impl+unit.

## C. Diagnosable join → `REQ-JOIN-DIAGNOSTICS`

- **Live progress** during the meet (replace the one-shot "Searching…" at `cli.rs:6268` with periodic
  elapsed / deadline output) so silence never reads as a hang.
- **Detailed failure** on meet-exhaustion — rendezvous candidates + families attempted (IPv4/IPv6),
  relay vs direct, the last concrete error — surfaced **before** any code prompt. (`connect_seed_holder`
  `pairhost.rs:437` and `dial_via_rendezvous` `meet.rs:281` currently swallow per-attempt errors —
  thread the last error up with attempt context.)
- **Propagate the terminal event:** `brain.rs:1024` `_ => continue` must deliver a daemon
  `NoSeedHolder`/`PairFail` to the CLI as a printed error, not discard it (this is *why the user saw
  nothing*).
- **`--verbose` / `SPT_LOG` discovery trace:** per-probe derived id, discovery path taken (mDNS / n0
  DNS / relay), per-family timeouts — opt-in, so the next WAN-join issue is self-serve. No knob exists
  today.
- Activate impl+unit.

## Cross-cutting

- **CONTEXT.md** corrected (done this session): **TOTP-epoch** (public, routes the meet) vs
  **TOTP-code** (secret, SPAKE2 only); _Avoid_ "the code routes discovery".
- **ADR-0030** doc-ratifies the cluster. Traceability: seed all three `REQ-*` with `required_stages =
  ["doc"]` (ADR + this doc are the doc evidence); impl/unit/int activate at the build milestone (rule 5).
- **Known kin (the other two silent join traps):** `docs/DEFERRED.md` row 23 (Windows-firewall inbound
  UDP drop) and row 30 (clock skew → wrong epoch). Both are refuted-but-related; the diagnostics (C)
  would have made each self-evident.
- **Public docs** (ride the build): MANIFEST/reference unaffected (no manifest surface); gh-pages
  `networking/overview` + a join/pairing troubleshooting note (families, `SPT_DISABLE_IPV6`, `--verbose`).


Now W4 site + traceable seeds + iroh single-family API check.