HERTZ -> DOYLE: v0.35.0 MESH-RECOVERY FIELD ACCEPTANCE TRIPLE 2026-07-17 HFENDULEAM + KITSUBITO Versions ======== HFENDULEAM spt 0.35.0, broker_image 0.35.0. KITSUBITO spt 0.35.0, broker_image 0.35.0. Remote update's systemd stop leg returned exit 1 after installing; `spt daemon start` reported already running, and subsequent status proved broker_image 0.35.0. No node remained half-updated. LEG 1 — POISON-ROW REPAIR: PASS ================================ HFENDULEAM pre-upgrade: - outer key 5ff50e75cd91d3b88ae3a0aa863e4371ad810c9747a43ac6242661e8cf8738ce - inner address.id ecb39e40e3400d46ddca73914685fb91b2d82969119e42293408511e27e55657 First v0.35 daemon load repaired the row from the validated roster: - outer key == inner id == 5ff50e75...8738ce - repaired row contains the roster relay/direct paths. - File was not bare-deleted. Existing valid rows remained; normal live peer progress added/enriched c893, cfd, and ecb rows. KITSUBITO pre-upgrade: - outer key 39c018a4...eabeab1 - inner address.id 14efb80c...c85f2d First v0.35 daemon load repaired it: - outer key == inner id == 39c018a4...eabeab1 - repaired row matches validated roster paths. - Existing valid 14ef row remained and was naturally enriched by live progress. Artifacts: - mesh-v035-hfenduleam-peer-addrs.pre.json - mesh-v035-hfenduleam-peer-addrs.post.json - mesh-v035-kitsubito-peer-addrs.pre.json - mesh-v035-kitsubito-peer-addrs.post.json LEG 2 — HEALTH TRUTH: PASS ========================== Controlled outage: suspended HFENDULEAM's stable daemon process while KITSUBITO remained live and sampled every 2 seconds. This removed KITSUBITO's only reachable peer; two other roster peers were already unreachable. JSON during outage: - peers_total: 3 - live_peers: 0 - all_fail_since_ms present - failing map names all three peers - stage for each: quic-connect - peer_health_verdict: degraded - pump heartbeat remained fresh, but view was NOT green. Human `spt subnet status` during outage, samples 1–7: peer reachability: DEGRADED — all 3/3 peers failing for 58s...70s stage: quic-connect last successful dial: 98s...110s ago After the suspended daemon resumed, sample 8 changed to healthy ONLY when real progress occurred: peer reachability: healthy (1/3 peers connected; last successful dial 1s ago) The verdict did not clear on heartbeat or elapsed time. The JSON later showed live_peers=1 and a newer last_dial_ok_ms. Artifacts: - mesh-v035-health-suspend.log - mesh-v035-human-health-outage.log Note: local stall_evict_count became 1 from the intentionally suspended process. This is induced test residue and the declared process-lifetime benign counter class, not a production failure. LEG 3 — ROUTE-CHAIN TRAP: PASS (field best-effort caveat recorded) ================================================================= KITSUBITO roster held a validated HFENDULEAM address whose address.id exactly matched pubkey 14efb80c...c85f2d. Probe sequence: 1. Established remote probe before outage. 2. Suspended HFENDULEAM daemon transport. 3. Removed only KITSUBITO cache key 14efb80c...c85f2d; backup saved at /tmp/mesh-v035-peer-addrs-before-trap.json. 4. Verified the row remained absent during the peer outage; no whole-file deletion or other state surgery. 5. Resumed HFENDULEAM. 6. Without restoring/editing the cache, KITSUBITO reconnected and wrote back a valid matching row: outer key == address.id == 14efb80c...c85f2d direct paths 100.68.35.65:65441 and 192.168.1.81:65441 7. KITSUBITO status returned live_peers=1, peer_health_verdict=healthy, with last_dial_ok_ms advanced. This proves absent-cache automatic convergence and zero operator repair. Field caveat: production offers no switch to disable only iroh id-only discovery, so the id-only resolver was not independently disabled after transport resumed. During the drop/outage the peer was black-holed; the deterministic mesh_recovery production-pump test supplies the stricter discovery-disabled arm. The field result is therefore the requested best-effort complement, not a claim that production id-only was separately instrumented off. Artifacts: - mesh-v035-trap-recovery.log - /tmp/mesh-v035-peer-addrs-before-trap.json on KITSUBITO - mesh-v035-trap-recovery-late.json where available; final cache/status captured directly. OVERALL: 3/3 behavioral PASS. - Poison rows repaired selectively on both incident nodes. - All-peer outage rendered DEGRADED with stage/count/timestamps despite fresh heartbeat, then recovered only on real peer progress. - Cache-row trap converged automatically with no repair surgery; strict discovery-disable is covered hermetically and was not exposed as a production field switch.