1	//! Inbound net-stream dispatch (D9-1) — the daemon-side acceptor that turns
2	//! the hand-wired serve loops of D5–D8 into production behavior.
3	//!
4	//! Every E2E test so far drove the serve half manually (a probe brain calling
5	//! `serve_sync` / `serve_update` / `apply_notif_feed` / `receive_wan` against
6	//! a known stream). In production nothing did — a peer could dial and write
7	//! and the bytes sat in the broker's stream log unread. This module is the
8	//! missing consumer: a daemon-hosted loop that watches the broker's stream
9	//! table for **peer-initiated** streams, classifies each by its first NDJSON
10	//! record, and hands it to the right serve loop on its own worker thread.
11	//!
12	//! ## Direction (the gotcha-#9 fact, now first-class)
13	//!
14	//! The stream table holds locally-opened outbound streams too. The dispatcher
15	//! claims only rows with `initiated_locally == false` — serving our own
16	//! requester stream would deadlock the requester against ourselves. The flag
17	//! is broker-recorded at registration (`open_stream` vs the accept loop),
18	//! never inferred.
19	//!
20	//! ## Classification (first-record demux)
21	//!
22	//! Streams are single-purpose; the opener's first record names the protocol.
23	//! The kind-tagged families collide on variant names (`request` is both
24	//! sync's and attach's opener; `chunk`/`done`/`err` recur everywhere), so the
25	//! demux keys on the first line's **shape**, not its kind alone:
26	//!
27	//! | first line | family |
28	//! |---|---|
29	//! | `kind=request` + `sync_id` | sync pull ([`crate::sync::serve_sync`]) |
30	//! | `kind=request` + `session_id` | remote-drive attach (D9-1-3) |
31	//! | `kind=request` + `endpoint`+`event` | remote rest op ([`crate::resthost::serve_rest`]) |
32	//! | `kind=request` + `owner`+`shell_ref` | cross-node shell link ([`crate::linkhost::serve_shell_link`]) |
33	//! | `kind=query` | update pull ([`crate::propagate::serve_update`]) |
34	//! | `kind=row` | notif feed ([`crate::notifsync::apply_notif_feed`]) |
35	//! | `kind=fetch` / `kind=push` | file transfer (D9-1-3) |
36	//! | untagged + `target`+`op_id` | WAN message feed ([`crate::wan::receive_wan`]) |
37	//! | untagged + `subnet`+`instance` | registry feed (D9-1-2) |
38	//! | anything else | unknown — dropped, loud |
39	//!
40	//! A family whose production wiring lands in a later slice is still
41	//! classified here (the table is complete) but routed to a typed
42	//! [`Unwired`](DispatchOutcome::Unwired) drop — loud, never silent.
43	//!
44	//! ## Worker model
45	//!
46	//! Each claimed stream gets its own worker thread with its **own** brain IPC
47	//! connection (the broker is multi-client by design; the probe-brain pattern
48	//! every E2E test uses). The peek and the serve use **separate** connections:
49	//! the peek brain subscribes from 0 to read the first line and is dropped;
50	//! the serve loop then subscribes from 0 on a fresh connection so it owns its
51	//! stream's event feed alone (no double-subscription interleave). Origin is
52	//! the table's `remote_id_hex` — handshake-proven, never payload
53	//! (REQ-HAZARD-WAN-ORIGIN-AUTH); every family's own gate then runs unchanged.
54	// [impl->REQ-NET-1]
55	// [impl->REQ-HAZARD-WAN-ORIGIN-AUTH]
56
57	use std::collections::HashSet;
58	use std::io;
59	use std::path::PathBuf;
60	use std::sync::atomic::{AtomicBool, Ordering};
61	use std::sync::Arc;
62	use std::thread;
63	use std::time::Duration;
64
65	use spt_net::net::notif::NotifDecoder;
66	use spt_net::net::replicate::{
67	    FeedDecoder as RegistryDecoder, NodeLabelUpdate, RegistryFeedRecord, RegistryUpdate,
68	};
69	use spt_net::net::wanmsg::WanDecoder;
70	use spt_store::contextstore::ContextStore;
71	use spt_store::notif::NotifStore;
72	use spt_store::perch;
73	use spt_store::roster::RosterStore;
74
75	use crate::attach::serve_attach;
76	use crate::brain::{Brain, BrokerEvent};
77	use crate::notifsync::{apply_notif_feed, NotifPolicy};
78	use crate::propagate::serve_update;
79	use crate::registryhost::{RegistryGatePolicy, RegistryHost};
80	use crate::relcache::ReleaseCache;
81	use crate::sync::{serve_sync, SyncPolicy};
82	use crate::wan::receive_wan;
83	use crate::xfer::serve_xfer;
84
85	/// Poll cadence for the stream-table watch (the dispatcher's only busy-wait;
86	/// workers block on their stream's event feed).
87	pub const DEFAULT_DISPATCH_POLL: Duration = Duration::from_millis(100);
88
89	/// What protocol a peer-initiated stream speaks, by its first record's shape.
90	#[derive(Debug, Clone, Copy, PartialEq, Eq)]
91	pub enum StreamFamily {
92	    /// A context sync pull (`SyncRecord::Request`).
93	    Sync,
94	    /// An update pull (`UpdRecord::Query`).
95	    Update,
96	    /// A notif replication feed (`NotifRecord::Row`).
97	    Notif,
98	    /// A WAN message feed (untagged `WanMessage` lines).
99	    WanMsg,
100	    /// A registry replication feed (untagged `RegistryUpdate` lines).
101	    Registry,
102	    /// A remote-drive attach (`AttachRecord::Request`).
103	    Attach,
104	    /// A file transfer (`XferRecord::Fetch`/`Push`).
105	    Xfer,
106	    /// A remote rest op (`RestRecord::Request` — suspend/wake from another
107	    /// node, M5-D5b).
108	    Rest,
109	    /// A cross-node shell-link op (`ShellLinkRecord::Request` — relink/cmd
110	    /// against an owner's shell on this node, M5-D8c).
111	    ShellLink,
112	    /// A subnet-scoped liveness probe (`ServeProbeRecord::Request` — "serving
113	    /// subnet X?", the `--nodes` view's detach-aware refinement, REQ-SUBNET-5).
114	    ServeProbe,
115	    /// Unclassifiable — dropped.
116	    Unknown,
117	}
118
119	/// Classify one decoded first line by shape (module-docs demux table).
120	pub fn classify_first_line(line: &serde_json::Value) -> StreamFamily {
121	    match line.get("kind").and_then(|k| k.as_str()) {
122	        Some("request") if line.get("sync_id").is_some() => StreamFamily::Sync,
123	        Some("request") if line.get("session_id").is_some() => StreamFamily::Attach,
124	        Some("request") if line.get("endpoint").is_some() && line.get("event").is_some() => {
125	            StreamFamily::Rest
126	        }
127	        Some("request") if line.get("owner").is_some() && line.get("shell_ref").is_some() => {
128	            StreamFamily::ShellLink
129	        }
130	        Some("request") if line.get("serve_probe").is_some() => StreamFamily::ServeProbe,
131	        Some("query") => StreamFamily::Update,
132	        // The status-only convergence query rides the update family (M8
133	        // decision 19, REQ-UPD-6) — same serve loop, no artifact bytes.
134	        Some("status_query") => StreamFamily::Update,
135	        Some("row") => StreamFamily::Notif,
136	        Some("fetch") | Some("push") => StreamFamily::Xfer,
137	        Some(_) => StreamFamily::Unknown,
138	        None => {
139	            if line.get("target").is_some() && line.get("op_id").is_some() {
140	                StreamFamily::WanMsg
141	            } else if line.get("subnet").is_some() && line.get("instance").is_some() {
142	                StreamFamily::Registry
143	            } else if line.get("subnet").is_some()
144	                && line.get("node").is_some()
145	                && line.get("label").is_some()
146	            {
147	                // A node-level label feed (REQ-SUBNET-3): an endpoint-less peer
148	                // sends label records first (no instance rows), so the feed's
149	                // first line is a NodeLabelUpdate. Same serve arm, same gate.
150	                StreamFamily::Registry
151	            } else {
152	                StreamFamily::Unknown
153	            }
154	        }
155	    }
156	}
157
158	/// Why a dispatched stream's worker ended — loud + typed, never silent.
159	#[derive(Debug)]
160	pub enum DispatchOutcome {
161	    /// The family's serve loop ran to its own outcome (rendered for the log).
162	    Served(String),
163	    /// The first line never arrived / didn't parse — dropped.
164	    Unknown,
165	    /// The worker failed (IO against broker or store).
166	    Failed(String),
167	}
168
169	/// The SPT_HOME-anchored paths the serve arms need. One value, cloned per
170	/// worker (policies/stores re-load from the canonical paths per request —
171	/// the access-gate stance, config edits take effect without a restart).
172	#[derive(Debug, Clone)]
173	pub struct DispatchPaths {
174	    /// The perch tree (WAN message funnel target).
175	    pub owlery: PathBuf,
176	    /// The staged-release cache dir ([`ReleaseCache::open`]).
177	    pub releases: PathBuf,
178	    /// Transient bundle/artifact scratch.
179	    pub scratch: PathBuf,
180	}
181
182	impl DispatchPaths {
183	    /// The production layout under [`perch::spt_home`].
184	    pub fn production() -> DispatchPaths {
185	        let home = perch::spt_home();
186	        DispatchPaths {
187	            owlery: perch::owlery_dir(),
188	            releases: home.join("releases"),
189	            scratch: home.join("scratch").join("net"),
190	        }
191	    }
192	}
193
194	/// Watch the broker's stream table and dispatch every **peer-initiated**
195	/// stream to a family worker, until `stop` is set. Blocks the calling thread
196	/// (the daemon spawns it on its own); workers detach (each ends with its
197	/// stream).
198	pub fn run_dispatch_loop(
199	    broker_name: &str,
200	    paths: &DispatchPaths,
201	    registry: Arc<RegistryHost>,
202	    stop: &AtomicBool,
203	    poll: Duration,
204	) -> io::Result<()> {
205	    let mut brain = connect(broker_name)?;
206	    let mut claimed: HashSet<u64> = HashSet::new();
207
208	    while !stop.load(Ordering::Relaxed) {
209	        let reply = brain.net_streams()?;
210	        for s in reply.streams {
211	            // Claim only peer-initiated rows with bytes (a QUIC stream is
212	            // invisible until its first write anyway — gotcha #6 — but the
213	            // next_seq guard also skips a row whose pump hasn't appended yet).
214	            if s.initiated_locally || s.next_seq == 0 || !claimed.insert(s.stream_id) {
215	                continue;
216	            }
217	            let name = broker_name.to_string();
218	            let origin = s.remote_id_hex.clone();
219	            let stream_id = s.stream_id;
220	            let p = paths.clone();
221	            let reg = Arc::clone(&registry);
222	            thread::spawn(move || {
223	                let outcome = worker(&name, stream_id, &origin, &p, &reg);
224	                match outcome {
225	                    DispatchOutcome::Served(_) => {}
226	                    other => eprintln!("DISPATCH:{stream_id}:{other:?}"),
227	                }
228	            });
229	        }
230	        thread::sleep(poll);
231	    }
232	    Ok(())
233	}
234
235	/// Connect a fresh brain IPC client (retrying through broker startup).
236	fn connect(broker_name: &str) -> io::Result<Brain> {
237	    let mut last = io::Error::other("broker never answered");
238	    for _ in 0..300 {
239	        match Brain::cold_start(broker_name, now_ms()) {
240	            Ok(b) => return Ok(b),
241	            Err(e) => last = e,
242	        }
243	        thread::sleep(Duration::from_millis(10));
244	    }
245	    Err(last)
246	}
247
248	fn now_ms() -> u64 {
249	    std::time::SystemTime::now()
250	        .duration_since(std::time::UNIX_EPOCH)
251	        .map(|d| d.as_millis() as u64)
252	        .unwrap_or(0)
253	}
254
255	/// Peek the stream's first NDJSON line on a throwaway brain connection.
256	fn peek_first_line(broker_name: &str, stream_id: u64) -> io::Result<Option<serde_json::Value>> {
257	    let mut brain = connect(broker_name)?;
258	    brain.net_stream_subscribe(stream_id, 0)?;
259	    let mut buf: Vec<u8> = Vec::new();
260	    loop {
261	        match brain.read_event()? {
262	            BrokerEvent::NetStreamData {
263	                stream_id: sid,
264	                bytes,
265	                ..
266	            } if sid == stream_id => {
267	                buf.extend_from_slice(&bytes);
268	                if let Some(nl) = buf.iter().position(|&b| b == b'\n') {
269	                    return Ok(serde_json::from_slice(&buf[..nl]).ok());
270	                }
271	            }
272	            BrokerEvent::NetStreamEof { stream_id: sid, .. } if sid == stream_id => {
273	                // Finished without a complete line: classify what we have
274	                // (a line without a trailing newline before EOF still counts).
275	                return Ok(serde_json::from_slice(&buf).ok());
276	            }
277	            BrokerEvent::Error { message } => return Err(io::Error::other(message)),
278	            _ => {}
279	        }
280	    }
281	}
282
283	/// One claimed stream's worker: peek → classify → run the family's serve
284	/// loop on a fresh brain connection.
285	fn worker(
286	    broker_name: &str,
287	    stream_id: u64,
288	    origin: &str,
289	    paths: &DispatchPaths,
290	    registry: &RegistryHost,
291	) -> DispatchOutcome {
292	    let line = match peek_first_line(broker_name, stream_id) {
293	        Ok(Some(v)) => v,
294	        Ok(None) => return DispatchOutcome::Unknown,
295	        Err(e) => return DispatchOutcome::Failed(e.to_string()),
296	    };
297	    let family = classify_first_line(&line);
298
299	    let mut brain = match connect(broker_name) {
300	        Ok(b) => b,
301	        Err(e) => return DispatchOutcome::Failed(e.to_string()),
302	    };
303
304	    match family {
305	        StreamFamily::Sync => {
306	            let cs = match ContextStore::open_or_init() {
307	                Ok(cs) => cs,
308	                Err(e) => return DispatchOutcome::Failed(e.to_string()),
309	            };
310	            let policy = SyncPolicy::load();
311	            match serve_sync(
312	                &mut brain,
313	                stream_id,
314	                origin,
315	                0,
316	                &cs,
317	                &policy,
318	                &paths.scratch,
319	            ) {
320	                Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
321	                Err(e) => DispatchOutcome::Failed(e.to_string()),
322	            }
323	        }
324	        StreamFamily::Update => {
325	            let cache = ReleaseCache::open(&paths.releases);
326	            let roster = RosterStore::load();
327	            match serve_update(&mut brain, stream_id, origin, 0, &cache, &roster) {
328	                Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
329	                Err(e) => DispatchOutcome::Failed(e.to_string()),
330	            }
331	        }
332	        StreamFamily::Notif => serve_notif_feed(&mut brain, stream_id, origin),
333	        StreamFamily::WanMsg => serve_wan_feed(&mut brain, stream_id, origin, paths),
334	        StreamFamily::Registry => serve_registry_feed(&mut brain, stream_id, origin, registry),
335	        StreamFamily::Attach => {
336	            // The gate subject is the session's OWNING endpoint, resolved
337	            // from the broker's table (the spawn's label) — never from wire
338	            // bytes. An unlabeled or unknown session refuses, fail-closed.
339	            let session_id = line.get("session_id").and_then(|v| v.as_u64());
340	            let endpoint = session_id.and_then(|sid| {
341	                brain.sessions().ok().and_then(|reply| {
342	                    reply
343	                        .sessions
344	                        .iter()
345	                        .find(|s| s.session_id == sid && !s.endpoint.is_empty())
346	                        .map(|s| s.endpoint.clone())
347	                })
348	            });
349	            match endpoint {
350	                Some(endpoint) => match serve_attach(&mut brain, stream_id, &endpoint, origin, 0) {
351	                    Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
352	                    Err(e) => DispatchOutcome::Failed(e.to_string()),
353	                },
354	                None => {
355	                    // Refuse politely: finish our send side, claim stays.
356	                    let _ = brain.net_stream_send(stream_id, &[], None, true);
357	                    DispatchOutcome::Served("attach refused: unlabeled/unknown session".into())
358	                }
359	            }
360	        }
361	        StreamFamily::Xfer => {
362	            // The wire names whose whitelist gates (the rule set); the origin
363	            // it is checked against stays handshake-proven. Blank refuses.
364	            let endpoint = line
365	                .get("endpoint")
366	                .and_then(|v| v.as_str())
367	                .filter(|e| !e.is_empty())
368	                .map(str::to_string);
369	            match endpoint {
370	                Some(endpoint) => {
371	                    let root = spt_store::xfer::transfers_dir();
372	                    let progress = paths.scratch.join("xfer-progress");
373	                    match serve_xfer(
374	                        &mut brain, stream_id, &endpoint, origin, 0, &root, &progress,
375	                    ) {
376	                        Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
377	                        Err(e) => DispatchOutcome::Failed(e.to_string()),
378	                    }
379	                }
380	                None => {
381	                    let _ = brain.net_stream_send(stream_id, &[], None, true);
382	                    DispatchOutcome::Served("xfer refused: no endpoint named".into())
383	                }
384	            }
385	        }
386	        StreamFamily::Rest => {
387	            // The wire names whose whitelist gates (the xfer posture); the
388	            // origin stays handshake-proven. The serve applies the event
389	            // through the LOCAL transition host — echo gate, wake effects,
390	            // and shell cascade run on this node, where the instance lives.
391	            // [impl->REQ-INST-6]
392	            let knob = crate::config::DaemonConfig::load().auto_suspend_after_ms;
393	            match crate::resthost::serve_rest(&mut brain, stream_id, origin, knob) {
394	                Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
395	                Err(e) => DispatchOutcome::Failed(e.to_string()),
396	            }
397	        }
398	        StreamFamily::ShellLink => {
399	            // The wire names the owner whose whitelist gates AND whose shell
400	            // tree resolves (owner exclusivity = resolution scope); the
401	            // action runs against the LOCAL owlery — perch, manifest, spool,
402	            // and binary all live here, the shell's node.
403	            // [impl->REQ-SHELL-2]
404	            match crate::linkhost::serve_shell_link(&mut brain, stream_id, origin, &paths.owlery) {
405	                Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
406	                Err(e) => DispatchOutcome::Failed(e.to_string()),
407	            }
408	        }
409	        StreamFamily::ServeProbe => {
410	            // No gate (serveprobe module docs): the answer is public
411	            // serve-state for a named subnet, read fresh from this node's
412	            // own stores — the QUIC handshake is the only subject.
413	            // [impl->REQ-SUBNET-5]
414	            match crate::serveprobe::serve_subnet_probe(&mut brain, stream_id) {
415	                Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
416	                Err(e) => DispatchOutcome::Failed(e.to_string()),
417	            }
418	        }
419	        StreamFamily::Unknown => DispatchOutcome::Unknown,
420	    }
421	}
422
423	/// Registry feed arm: decode `RegistryUpdate` lines, merge each batch into
424	/// the daemon's [`RegistryHost`] under the fail-closed gate (member subnet ∧
425	/// origin trusted in that subnet), until the peer finishes.
426	///
427	/// An admitted batch may carry **attention flips** (REQ-INST-3): a peer
428	/// instance of an id observed transitioning to `Active` — its node's daemon
429	/// just gave it a driver, so a *locally hosted* instance of the same id
430	/// rests warm (CONTEXT: driving `ling@laptop` makes `ling@desktop`
431	/// dormant). The rest edge's transition echo is gate-armed
432	/// ([`crate::resting::arm_transition_echo`]) — the endpoint's own pulse
433	/// loop runs the bounded summarizer call, never this worker (the KH 7.4
434	/// non-blocking posture: N flips in one feed are N gate files, not N LLM
435	/// calls serialized here).
436	// [impl->REQ-INST-7]
437	fn serve_registry_feed(
438	    brain: &mut Brain,
439	    stream_id: u64,
440	    origin: &str,
441	    registry: &RegistryHost,
442	) -> DispatchOutcome {
443	    let policy = RegistryGatePolicy::load();
444	    if let Err(e) = brain.net_stream_subscribe(stream_id, 0) {
445	        return DispatchOutcome::Failed(e.to_string());
446	    }
447	    let mut decoder = RegistryDecoder::new();
448	    let mut merged = 0usize;
449	    loop {
450	        match brain.read_event() {
451	            Ok(BrokerEvent::NetStreamData {
452	                stream_id: sid,
453	                bytes,
454	                ..
455	            }) if sid == stream_id => {
456	                let records = decoder.push(&bytes);
457	                if records.is_empty() {
458	                    continue;
459	                }
460	                // Split the feed into instance rows and node-level labels
461	                // (REQ-SUBNET-3) — each goes through its own gated apply path.
462	                let mut updates: Vec<RegistryUpdate> = Vec::new();
463	                let mut labels: Vec<NodeLabelUpdate> = Vec::new();
464	                for rec in records {
465	                    match rec {
466	                        RegistryFeedRecord::Instance(u) => updates.push(u),
467	                        RegistryFeedRecord::NodeLabel(n) => labels.push(n),
468	                    }
469	                }
470	                if !labels.is_empty() {
471	                    merged += registry.apply_node_labels(origin, &labels, &policy).len();
472	                }
473	                if updates.is_empty() {
474	                    continue;
475	                }
476	                let (verdicts, flips) = registry.apply_feed_flips(origin, &updates, &policy);
477	                merged += verdicts.len();
478	                // Attention shift (D9-2): only ids this node actually hosts
479	                // (a perch with a record); everything else is a no-op here.
480	                // [impl->REQ-INST-3]
481	                for id in flips {
482	                    let local = perch::resolve_perch_path(&id, spt_store::perch::ParentHint::Infer);
483	                    if local.join("info.json").exists() {
484	                        if let Err(e) = crate::resting::daemon_rest_event(
485	                            &id,
486	                            crate::resting::RestEvent::AttentionShift,
487	                            None,
488	                        ) {
489	                            eprintln!("REST_SHIFT_FAIL:{id}:{e}");
490	                        }
491	                    }
492	                }
493	            }
494	            Ok(BrokerEvent::NetStreamEof { stream_id: sid, .. }) if sid == stream_id => {
495	                return DispatchOutcome::Served(format!("registry feed: {merged} record(s)"));
496	            }
497	            Ok(BrokerEvent::Error { message }) => return DispatchOutcome::Failed(message),
498	            Ok(_) => {}
499	            Err(e) => return DispatchOutcome::Failed(e.to_string()),
500	        }
501	    }
502	}
503
504	/// Notif feed arm: decode rows as they arrive, apply each batch under the
505	/// fail-closed gate, until the peer finishes. The policy re-loads per feed
506	/// (not per record) — the same freshness the other arms get per request.
507	///
508	/// A **freshly inserted** undismissed row additionally gets a first-fire here
509	/// (REQ-PRES-1, the convergent cross-node redirect's receiving half): the
510	/// producing node skipped its local surface because the MRA winner lives
511	/// here — this node's own presence resolution surfaces it. A row this node's
512	/// MRA also calls remote stays unmarked and rides on (someone else's
513	/// feed-apply surfaces it); replays merge `Unchanged` and never re-fire.
514	// [impl->REQ-NOTIF-1]
515	// [impl->REQ-PRES-1]
516	fn serve_notif_feed(brain: &mut Brain, stream_id: u64, origin: &str) -> DispatchOutcome {
517	    let store = match NotifStore::open() {
518	        Ok(s) => s,
519	        Err(e) => return DispatchOutcome::Failed(e.to_string()),
520	    };
521	    let policy = NotifPolicy::load();
522	    // This serving node's identity for the presence resolution: the broker's
523	    // handshake identity (the same hex peers see in our registry rows).
524	    let self_hex = brain
525	        .net_status()
526	        .ok()
527	        .and_then(|s| s.node_id_hex)
528	        .unwrap_or_default();
529	    if let Err(e) = brain.net_stream_subscribe(stream_id, 0) {
530	        return DispatchOutcome::Failed(e.to_string());
531	    }
532	    let mut decoder = NotifDecoder::new();
533	    let mut applied = 0usize;
534	    loop {
535	        match brain.read_event() {
536	            Ok(BrokerEvent::NetStreamData {
537	                stream_id: sid,
538	                bytes,
539	                ..
540	            }) if sid == stream_id => {
541	                let records = decoder.push(&bytes);
542	                if records.is_empty() {
543	                    continue;
544	                }
545	                match apply_notif_feed(&store, origin, &records, &policy) {
546	                    Ok(verdicts) => {
547	                        applied += verdicts.len();
548	                        surface_fresh_rows(&store, &records, &verdicts, &self_hex);
549	                    }
550	                    Err(e) => return DispatchOutcome::Failed(e),
551	                }
552	            }
553	            Ok(BrokerEvent::NetStreamEof { stream_id: sid, .. }) if sid == stream_id => {
554	                return DispatchOutcome::Served(format!("notif feed: {applied} record(s)"));
555	            }
556	            Ok(BrokerEvent::Error { message }) => return DispatchOutcome::Failed(message),
557	            Ok(_) => {}
558	            Err(e) => return DispatchOutcome::Failed(e.to_string()),
559	        }
560	    }
561	}
562
563	/// The receiving half of the cross-node notif redirect (REQ-PRES-1): first-
564	/// fire every **freshly inserted** undismissed row from an applied feed
565	/// batch, under this node's own presence resolution. Best-effort and loud —
566	/// a failed surface costs a boundary-resurface delay, never the feed.
567	// [impl->REQ-PRES-1]
568	fn surface_fresh_rows(
569	    store: &NotifStore,
570	    records: &[spt_net::net::notif::NotifRecord],
571	    verdicts: &[crate::notifsync::NotifApplyVerdict],
572	    self_hex: &str,
573	) {
574	    use crate::notifsync::NotifApplyVerdict;
575	    use spt_store::notif::MergeOutcome;
576	    let fresh = records.iter().zip(verdicts).filter_map(|(rec, v)| {
577	        let spt_net::net::notif::NotifRecord::Row { row } = rec;
578	        (matches!(v, NotifApplyVerdict::Applied(MergeOutcome::Inserted)) && !row.dismissed)
579	            .then_some(row)
580	    });
581	    let policy = crate::notif::NotifSurfacePolicy::load();
582	    let regs = crate::presence::load_registry_snapshots(&crate::presence::registry_snapshot_dir());
583	    let owlery = perch::owlery_dir();
584	    for row in fresh {
585	        match crate::notif::first_fire_at(store, row, &policy, &owlery, self_hex, &regs, now_ms()) {
586	            Ok(_) => {}
587	            Err(e) => eprintln!("NOTIF_FEED_SURFACE_FAIL:{}: {e}", row.notif_id),
588	        }
589	    }
590	}
591
592	/// WAN message arm: decode `WanMessage` lines, funnel each through the
593	/// access-gate + op-id-dedup path, until the peer finishes.
594	// [impl->REQ-NET-1]
595	fn serve_wan_feed(
596	    brain: &mut Brain,
597	    stream_id: u64,
598	    origin: &str,
599	    paths: &DispatchPaths,
600	) -> DispatchOutcome {
601	    if let Err(e) = brain.net_stream_subscribe(stream_id, 0) {
602	        return DispatchOutcome::Failed(e.to_string());
603	    }
604	    let mut decoder = WanDecoder::new();
605	    let mut handled = 0usize;
606	    loop {
607	        match brain.read_event() {
608	            Ok(BrokerEvent::NetStreamData {
609	                stream_id: sid,
610	                bytes,
611	                ..
612	            }) if sid == stream_id => {
613	                for msg in decoder.push(&bytes) {
614	                    let _ = receive_wan(&msg, origin, &paths.owlery);
615	                    handled += 1;
616	                }
617	            }
618	            Ok(BrokerEvent::NetStreamEof { stream_id: sid, .. }) if sid == stream_id => {
619	                return DispatchOutcome::Served(format!("wan feed: {handled} record(s)"));
620	            }
621	            Ok(BrokerEvent::Error { message }) => return DispatchOutcome::Failed(message),
622	            Ok(_) => {}
623	            Err(e) => return DispatchOutcome::Failed(e.to_string()),
624	        }
625	    }
626	}
627
628	/// Spawn the dispatcher on its own thread against the canonical broker
629	/// socket — the daemon-boot entry ([`crate::daemon::Daemon::run`]).
630	pub fn spawn_dispatcher(
631	    broker_name: String,
632	    registry: Arc<RegistryHost>,
633	    stop: Arc<AtomicBool>,
634	) -> thread::JoinHandle<()> {
635	    thread::spawn(move || {
636	        let paths = DispatchPaths::production();
637	        if let Err(e) =
638	            run_dispatch_loop(&broker_name, &paths, registry, &stop, DEFAULT_DISPATCH_POLL)
639	        {
640	            eprintln!("DISPATCH_LOOP_FAIL: {e}");
641	        }
642	    })
643	}
644
645	#[cfg(test)]
646	mod tests {
647	    use super::*;
648	    use serde_json::json;
649
650	    // [unit->REQ-NET-1] the demux table: every family's first record
651	    // classifies correctly, including the `request` kind collision.
652	    #[test]
653	    fn classifies_every_family_by_first_line_shape() {
654	        let sync = json!({"kind": "request", "sync_id": "pull-1", "refs": [], "have_tips": {}});
655	        assert_eq!(classify_first_line(&sync), StreamFamily::Sync);
656
657	        let attach = json!({"kind": "request", "session_id": 3, "from_seq": 0});
658	        assert_eq!(classify_first_line(&attach), StreamFamily::Attach);
659
660	        // [unit->REQ-INST-6] the rest op's opener disambiguates by shape too.
661	        let rest =
662	            json!({"kind": "request", "endpoint": "ling", "event": "suspend", "op_id": "a:1"});
663	        assert_eq!(classify_first_line(&rest), StreamFamily::Rest);
664
665	        let update =
666	            json!({"kind": "query", "upd_id": "u1", "current_version": 5, "channel": "stable"});
667	        assert_eq!(classify_first_line(&update), StreamFamily::Update);
668
669	        let notif = json!({"kind": "row", "row": {}});
670	        assert_eq!(classify_first_line(&notif), StreamFamily::Notif);
671
672	        let fetch = json!({"kind": "fetch", "path": "a.txt", "xfer_id": "x1"});
673	        assert_eq!(classify_first_line(&fetch), StreamFamily::Xfer);
674	        let push = json!({"kind": "push", "path": "a.txt", "total": 4, "xfer_id": "x2"});
675	        assert_eq!(classify_first_line(&push), StreamFamily::Xfer);
676
677	        let wan = json!({"target": "ling", "from": "oak", "body": "hi", "op_id": "w1"});
678	        assert_eq!(classify_first_line(&wan), StreamFamily::WanMsg);
679
680	        let reg = json!({"subnet": "home", "endpoint_id": "ling",
681	            "instance": {"node": "n1", "status": "Active", "epoch": 4}});
682	        assert_eq!(classify_first_line(&reg), StreamFamily::Registry);
683	    }
684
685	    // [unit->REQ-NET-1] unclassifiable shapes are Unknown — dropped loud,
686	    // never routed to a gate-bearing serve loop on a guess.
687	    #[test]
688	    fn unknown_shapes_refuse_classification() {
689	        for v in [
690	            json!({"kind": "chunk", "sync_id": "s", "offset": 0, "data_b64": ""}), // mid-protocol, not an opener
691	            json!({"kind": "request"}), // request without any discriminant
692	            json!({"kind": "request", "endpoint": "ling"}), // rest-ish but no event
693	            json!({"kind": "nonsense"}),
694	            json!({"target": "ling"}), // wan-ish but no op_id
695	            json!({"subnet": "home"}), // registry-ish but no instance
696	            json!({"completely": "other"}),
697	            json!(42),
698	        ] {
699	            assert_eq!(classify_first_line(&v), StreamFamily::Unknown, "for {v}");
700	        }
701	    }
702	}
703

//! Inbound net-stream dispatch (D9-1) — the daemon-side acceptor that turns
//! the hand-wired serve loops of D5–D8 into production behavior.
//!
//! Every E2E test so far drove the serve half manually (a probe brain calling
//! `serve_sync` / `serve_update` / `apply_notif_feed` / `receive_wan` against
//! a known stream). In production nothing did — a peer could dial and write
//! and the bytes sat in the broker's stream log unread. This module is the
//! missing consumer: a daemon-hosted loop that watches the broker's stream
//! table for **peer-initiated** streams, classifies each by its first NDJSON
//! record, and hands it to the right serve loop on its own worker thread.
//!
//! ## Direction (the gotcha-#9 fact, now first-class)
//!
//! The stream table holds locally-opened outbound streams too. The dispatcher
//! claims only rows with `initiated_locally == false` — serving our own
//! requester stream would deadlock the requester against ourselves. The flag
//! is broker-recorded at registration (`open_stream` vs the accept loop),
//! never inferred.
//!
//! ## Classification (first-record demux)
//!
//! Streams are single-purpose; the opener's first record names the protocol.
//! The kind-tagged families collide on variant names (`request` is both
//! sync's and attach's opener; `chunk`/`done`/`err` recur everywhere), so the
//! demux keys on the first line's **shape**, not its kind alone:
//!
//! | first line | family |
//! |---|---|
//! | `kind=request` + `sync_id` | sync pull ([`crate::sync::serve_sync`]) |
//! | `kind=request` + `session_id` | remote-drive attach (D9-1-3) |
//! | `kind=request` + `endpoint`+`event` | remote rest op ([`crate::resthost::serve_rest`]) |
//! | `kind=request` + `owner`+`shell_ref` | cross-node shell link ([`crate::linkhost::serve_shell_link`]) |
//! | `kind=query` | update pull ([`crate::propagate::serve_update`]) |
//! | `kind=row` | notif feed ([`crate::notifsync::apply_notif_feed`]) |
//! | `kind=fetch` / `kind=push` | file transfer (D9-1-3) |
//! | untagged + `target`+`op_id` | WAN message feed ([`crate::wan::receive_wan`]) |
//! | untagged + `subnet`+`instance` | registry feed (D9-1-2) |
//! | anything else | unknown — dropped, loud |
//!
//! A family whose production wiring lands in a later slice is still
//! classified here (the table is complete) but routed to a typed
//! [`Unwired`](DispatchOutcome::Unwired) drop — loud, never silent.
//!
//! ## Worker model
//!
//! Each claimed stream gets its own worker thread with its **own** brain IPC
//! connection (the broker is multi-client by design; the probe-brain pattern
//! every E2E test uses). The peek and the serve use **separate** connections:
//! the peek brain subscribes from 0 to read the first line and is dropped;
//! the serve loop then subscribes from 0 on a fresh connection so it owns its
//! stream's event feed alone (no double-subscription interleave). Origin is
//! the table's `remote_id_hex` — handshake-proven, never payload
//! (REQ-HAZARD-WAN-ORIGIN-AUTH); every family's own gate then runs unchanged.
// [impl->REQ-NET-1]
// [impl->REQ-HAZARD-WAN-ORIGIN-AUTH]

use std::collections::HashSet;
use std::io;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::thread;
use std::time::Duration;

use spt_net::net::notif::NotifDecoder;
use spt_net::net::replicate::{
    FeedDecoder as RegistryDecoder, NodeLabelUpdate, RegistryFeedRecord, RegistryUpdate,
};
use spt_net::net::wanmsg::WanDecoder;
use spt_store::contextstore::ContextStore;
use spt_store::notif::NotifStore;
use spt_store::perch;
use spt_store::roster::RosterStore;

use crate::attach::serve_attach;
use crate::brain::{Brain, BrokerEvent};
use crate::notifsync::{apply_notif_feed, NotifPolicy};
use crate::propagate::serve_update;
use crate::registryhost::{RegistryGatePolicy, RegistryHost};
use crate::relcache::ReleaseCache;
use crate::sync::{serve_sync, SyncPolicy};
use crate::wan::receive_wan;
use crate::xfer::serve_xfer;

/// Poll cadence for the stream-table watch (the dispatcher's only busy-wait;
/// workers block on their stream's event feed).
pub const DEFAULT_DISPATCH_POLL: Duration = Duration::from_millis(100);

/// What protocol a peer-initiated stream speaks, by its first record's shape.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum StreamFamily {
    /// A context sync pull (`SyncRecord::Request`).
    Sync,
    /// An update pull (`UpdRecord::Query`).
    Update,
    /// A notif replication feed (`NotifRecord::Row`).
    Notif,
    /// A WAN message feed (untagged `WanMessage` lines).
    WanMsg,
    /// A registry replication feed (untagged `RegistryUpdate` lines).
    Registry,
    /// A remote-drive attach (`AttachRecord::Request`).
    Attach,
    /// A file transfer (`XferRecord::Fetch`/`Push`).
    Xfer,
    /// A remote rest op (`RestRecord::Request` — suspend/wake from another
    /// node, M5-D5b).
    Rest,
    /// A cross-node shell-link op (`ShellLinkRecord::Request` — relink/cmd
    /// against an owner's shell on this node, M5-D8c).
    ShellLink,
    /// A subnet-scoped liveness probe (`ServeProbeRecord::Request` — "serving
    /// subnet X?", the `--nodes` view's detach-aware refinement, REQ-SUBNET-5).
    ServeProbe,
    /// Unclassifiable — dropped.
    Unknown,
}

/// Classify one decoded first line by shape (module-docs demux table).
pub fn classify_first_line(line: &serde_json::Value) -> StreamFamily {
    match line.get("kind").and_then(|k| k.as_str()) {
        Some("request") if line.get("sync_id").is_some() => StreamFamily::Sync,
        Some("request") if line.get("session_id").is_some() => StreamFamily::Attach,
        Some("request") if line.get("endpoint").is_some() && line.get("event").is_some() => {
            StreamFamily::Rest
        }
        Some("request") if line.get("owner").is_some() && line.get("shell_ref").is_some() => {
            StreamFamily::ShellLink
        }
        Some("request") if line.get("serve_probe").is_some() => StreamFamily::ServeProbe,
        Some("query") => StreamFamily::Update,
        // The status-only convergence query rides the update family (M8
        // decision 19, REQ-UPD-6) — same serve loop, no artifact bytes.
        Some("status_query") => StreamFamily::Update,
        Some("row") => StreamFamily::Notif,
        Some("fetch") | Some("push") => StreamFamily::Xfer,
        Some(_) => StreamFamily::Unknown,
        None => {
            if line.get("target").is_some() && line.get("op_id").is_some() {
                StreamFamily::WanMsg
            } else if line.get("subnet").is_some() && line.get("instance").is_some() {
                StreamFamily::Registry
            } else if line.get("subnet").is_some()
                && line.get("node").is_some()
                && line.get("label").is_some()
            {
                // A node-level label feed (REQ-SUBNET-3): an endpoint-less peer
                // sends label records first (no instance rows), so the feed's
                // first line is a NodeLabelUpdate. Same serve arm, same gate.
                StreamFamily::Registry
            } else {
                StreamFamily::Unknown
            }
        }
    }
}

/// Why a dispatched stream's worker ended — loud + typed, never silent.
#[derive(Debug)]
pub enum DispatchOutcome {
    /// The family's serve loop ran to its own outcome (rendered for the log).
    Served(String),
    /// The first line never arrived / didn't parse — dropped.
    Unknown,
    /// The worker failed (IO against broker or store).
    Failed(String),
}

/// The SPT_HOME-anchored paths the serve arms need. One value, cloned per
/// worker (policies/stores re-load from the canonical paths per request —
/// the access-gate stance, config edits take effect without a restart).
#[derive(Debug, Clone)]
pub struct DispatchPaths {
    /// The perch tree (WAN message funnel target).
    pub owlery: PathBuf,
    /// The staged-release cache dir ([`ReleaseCache::open`]).
    pub releases: PathBuf,
    /// Transient bundle/artifact scratch.
    pub scratch: PathBuf,
}

impl DispatchPaths {
    /// The production layout under [`perch::spt_home`].
    pub fn production() -> DispatchPaths {
        let home = perch::spt_home();
        DispatchPaths {
            owlery: perch::owlery_dir(),
            releases: home.join("releases"),
            scratch: home.join("scratch").join("net"),
        }
    }
}

/// Watch the broker's stream table and dispatch every **peer-initiated**
/// stream to a family worker, until `stop` is set. Blocks the calling thread
/// (the daemon spawns it on its own); workers detach (each ends with its
/// stream).
pub fn run_dispatch_loop(
    broker_name: &str,
    paths: &DispatchPaths,
    registry: Arc<RegistryHost>,
    stop: &AtomicBool,
    poll: Duration,
) -> io::Result<()> {
    let mut brain = connect(broker_name)?;
    let mut claimed: HashSet<u64> = HashSet::new();

    while !stop.load(Ordering::Relaxed) {
        let reply = brain.net_streams()?;
        for s in reply.streams {
            // Claim only peer-initiated rows with bytes (a QUIC stream is
            // invisible until its first write anyway — gotcha #6 — but the
            // next_seq guard also skips a row whose pump hasn't appended yet).
            if s.initiated_locally || s.next_seq == 0 || !claimed.insert(s.stream_id) {
                continue;
            }
            let name = broker_name.to_string();
            let origin = s.remote_id_hex.clone();
            let stream_id = s.stream_id;
            let p = paths.clone();
            let reg = Arc::clone(&registry);
            thread::spawn(move || {
                let outcome = worker(&name, stream_id, &origin, &p, &reg);
                match outcome {
                    DispatchOutcome::Served(_) => {}
                    other => eprintln!("DISPATCH:{stream_id}:{other:?}"),
                }
            });
        }
        thread::sleep(poll);
    }
    Ok(())
}

/// Connect a fresh brain IPC client (retrying through broker startup).
fn connect(broker_name: &str) -> io::Result<Brain> {
    let mut last = io::Error::other("broker never answered");
    for _ in 0..300 {
        match Brain::cold_start(broker_name, now_ms()) {
            Ok(b) => return Ok(b),
            Err(e) => last = e,
        }
        thread::sleep(Duration::from_millis(10));
    }
    Err(last)
}

fn now_ms() -> u64 {
    std::time::SystemTime::now()
        .duration_since(std::time::UNIX_EPOCH)
        .map(|d| d.as_millis() as u64)
        .unwrap_or(0)
}

/// Peek the stream's first NDJSON line on a throwaway brain connection.
fn peek_first_line(broker_name: &str, stream_id: u64) -> io::Result<Option<serde_json::Value>> {
    let mut brain = connect(broker_name)?;
    brain.net_stream_subscribe(stream_id, 0)?;
    let mut buf: Vec<u8> = Vec::new();
    loop {
        match brain.read_event()? {
            BrokerEvent::NetStreamData {
                stream_id: sid,
                bytes,
                ..
            } if sid == stream_id => {
                buf.extend_from_slice(&bytes);
                if let Some(nl) = buf.iter().position(|&b| b == b'\n') {
                    return Ok(serde_json::from_slice(&buf[..nl]).ok());
                }
            }
            BrokerEvent::NetStreamEof { stream_id: sid, .. } if sid == stream_id => {
                // Finished without a complete line: classify what we have
                // (a line without a trailing newline before EOF still counts).
                return Ok(serde_json::from_slice(&buf).ok());
            }
            BrokerEvent::Error { message } => return Err(io::Error::other(message)),
            _ => {}
        }
    }
}

/// One claimed stream's worker: peek → classify → run the family's serve
/// loop on a fresh brain connection.
fn worker(
    broker_name: &str,
    stream_id: u64,
    origin: &str,
    paths: &DispatchPaths,
    registry: &RegistryHost,
) -> DispatchOutcome {
    let line = match peek_first_line(broker_name, stream_id) {
        Ok(Some(v)) => v,
        Ok(None) => return DispatchOutcome::Unknown,
        Err(e) => return DispatchOutcome::Failed(e.to_string()),
    };
    let family = classify_first_line(&line);

    let mut brain = match connect(broker_name) {
        Ok(b) => b,
        Err(e) => return DispatchOutcome::Failed(e.to_string()),
    };

    match family {
        StreamFamily::Sync => {
            let cs = match ContextStore::open_or_init() {
                Ok(cs) => cs,
                Err(e) => return DispatchOutcome::Failed(e.to_string()),
            };
            let policy = SyncPolicy::load();
            match serve_sync(
                &mut brain,
                stream_id,
                origin,
                0,
                &cs,
                &policy,
                &paths.scratch,
            ) {
                Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
                Err(e) => DispatchOutcome::Failed(e.to_string()),
            }
        }
        StreamFamily::Update => {
            let cache = ReleaseCache::open(&paths.releases);
            let roster = RosterStore::load();
            match serve_update(&mut brain, stream_id, origin, 0, &cache, &roster) {
                Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
                Err(e) => DispatchOutcome::Failed(e.to_string()),
            }
        }
        StreamFamily::Notif => serve_notif_feed(&mut brain, stream_id, origin),
        StreamFamily::WanMsg => serve_wan_feed(&mut brain, stream_id, origin, paths),
        StreamFamily::Registry => serve_registry_feed(&mut brain, stream_id, origin, registry),
        StreamFamily::Attach => {
            // The gate subject is the session's OWNING endpoint, resolved
            // from the broker's table (the spawn's label) — never from wire
            // bytes. An unlabeled or unknown session refuses, fail-closed.
            let session_id = line.get("session_id").and_then(|v| v.as_u64());
            let endpoint = session_id.and_then(|sid| {
                brain.sessions().ok().and_then(|reply| {
                    reply
                        .sessions
                        .iter()
                        .find(|s| s.session_id == sid && !s.endpoint.is_empty())
                        .map(|s| s.endpoint.clone())
                })
            });
            match endpoint {
                Some(endpoint) => match serve_attach(&mut brain, stream_id, &endpoint, origin, 0) {
                    Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
                    Err(e) => DispatchOutcome::Failed(e.to_string()),
                },
                None => {
                    // Refuse politely: finish our send side, claim stays.
                    let _ = brain.net_stream_send(stream_id, &[], None, true);
                    DispatchOutcome::Served("attach refused: unlabeled/unknown session".into())
                }
            }
        }
        StreamFamily::Xfer => {
            // The wire names whose whitelist gates (the rule set); the origin
            // it is checked against stays handshake-proven. Blank refuses.
            let endpoint = line
                .get("endpoint")
                .and_then(|v| v.as_str())
                .filter(|e| !e.is_empty())
                .map(str::to_string);
            match endpoint {
                Some(endpoint) => {
                    let root = spt_store::xfer::transfers_dir();
                    let progress = paths.scratch.join("xfer-progress");
                    match serve_xfer(
                        &mut brain, stream_id, &endpoint, origin, 0, &root, &progress,
                    ) {
                        Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
                        Err(e) => DispatchOutcome::Failed(e.to_string()),
                    }
                }
                None => {
                    let _ = brain.net_stream_send(stream_id, &[], None, true);
                    DispatchOutcome::Served("xfer refused: no endpoint named".into())
                }
            }
        }
        StreamFamily::Rest => {
            // The wire names whose whitelist gates (the xfer posture); the
            // origin stays handshake-proven. The serve applies the event
            // through the LOCAL transition host — echo gate, wake effects,
            // and shell cascade run on this node, where the instance lives.
            // [impl->REQ-INST-6]
            let knob = crate::config::DaemonConfig::load().auto_suspend_after_ms;
            match crate::resthost::serve_rest(&mut brain, stream_id, origin, knob) {
                Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
                Err(e) => DispatchOutcome::Failed(e.to_string()),
            }
        }
        StreamFamily::ShellLink => {
            // The wire names the owner whose whitelist gates AND whose shell
            // tree resolves (owner exclusivity = resolution scope); the
            // action runs against the LOCAL owlery — perch, manifest, spool,
            // and binary all live here, the shell's node.
            // [impl->REQ-SHELL-2]
            match crate::linkhost::serve_shell_link(&mut brain, stream_id, origin, &paths.owlery) {
                Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
                Err(e) => DispatchOutcome::Failed(e.to_string()),
            }
        }
        StreamFamily::ServeProbe => {
            // No gate (serveprobe module docs): the answer is public
            // serve-state for a named subnet, read fresh from this node's
            // own stores — the QUIC handshake is the only subject.
            // [impl->REQ-SUBNET-5]
            match crate::serveprobe::serve_subnet_probe(&mut brain, stream_id) {
                Ok(out) => DispatchOutcome::Served(format!("{out:?}")),
                Err(e) => DispatchOutcome::Failed(e.to_string()),
            }
        }
        StreamFamily::Unknown => DispatchOutcome::Unknown,
    }
}

/// Registry feed arm: decode `RegistryUpdate` lines, merge each batch into
/// the daemon's [`RegistryHost`] under the fail-closed gate (member subnet ∧
/// origin trusted in that subnet), until the peer finishes.
///
/// An admitted batch may carry **attention flips** (REQ-INST-3): a peer
/// instance of an id observed transitioning to `Active` — its node's daemon
/// just gave it a driver, so a *locally hosted* instance of the same id
/// rests warm (CONTEXT: driving `ling@laptop` makes `ling@desktop`
/// dormant). The rest edge's transition echo is gate-armed
/// ([`crate::resting::arm_transition_echo`]) — the endpoint's own pulse
/// loop runs the bounded summarizer call, never this worker (the KH 7.4
/// non-blocking posture: N flips in one feed are N gate files, not N LLM
/// calls serialized here).
// [impl->REQ-INST-7]
fn serve_registry_feed(
    brain: &mut Brain,
    stream_id: u64,
    origin: &str,
    registry: &RegistryHost,
) -> DispatchOutcome {
    let policy = RegistryGatePolicy::load();
    if let Err(e) = brain.net_stream_subscribe(stream_id, 0) {
        return DispatchOutcome::Failed(e.to_string());
    }
    let mut decoder = RegistryDecoder::new();
    let mut merged = 0usize;
    loop {
        match brain.read_event() {
            Ok(BrokerEvent::NetStreamData {
                stream_id: sid,
                bytes,
                ..
            }) if sid == stream_id => {
                let records = decoder.push(&bytes);
                if records.is_empty() {
                    continue;
                }
                // Split the feed into instance rows and node-level labels
                // (REQ-SUBNET-3) — each goes through its own gated apply path.
                let mut updates: Vec<RegistryUpdate> = Vec::new();
                let mut labels: Vec<NodeLabelUpdate> = Vec::new();
                for rec in records {
                    match rec {
                        RegistryFeedRecord::Instance(u) => updates.push(u),
                        RegistryFeedRecord::NodeLabel(n) => labels.push(n),
                    }
                }
                if !labels.is_empty() {
                    merged += registry.apply_node_labels(origin, &labels, &policy).len();
                }
                if updates.is_empty() {
                    continue;
                }
                let (verdicts, flips) = registry.apply_feed_flips(origin, &updates, &policy);
                merged += verdicts.len();
                // Attention shift (D9-2): only ids this node actually hosts
                // (a perch with a record); everything else is a no-op here.
                // [impl->REQ-INST-3]
                for id in flips {
                    let local = perch::resolve_perch_path(&id, spt_store::perch::ParentHint::Infer);
                    if local.join("info.json").exists() {
                        if let Err(e) = crate::resting::daemon_rest_event(
                            &id,
                            crate::resting::RestEvent::AttentionShift,
                            None,
                        ) {
                            eprintln!("REST_SHIFT_FAIL:{id}:{e}");
                        }
                    }
                }
            }
            Ok(BrokerEvent::NetStreamEof { stream_id: sid, .. }) if sid == stream_id => {
                return DispatchOutcome::Served(format!("registry feed: {merged} record(s)"));
            }
            Ok(BrokerEvent::Error { message }) => return DispatchOutcome::Failed(message),
            Ok(_) => {}
            Err(e) => return DispatchOutcome::Failed(e.to_string()),
        }
    }
}

/// Notif feed arm: decode rows as they arrive, apply each batch under the
/// fail-closed gate, until the peer finishes. The policy re-loads per feed
/// (not per record) — the same freshness the other arms get per request.
///
/// A **freshly inserted** undismissed row additionally gets a first-fire here
/// (REQ-PRES-1, the convergent cross-node redirect's receiving half): the
/// producing node skipped its local surface because the MRA winner lives
/// here — this node's own presence resolution surfaces it. A row this node's
/// MRA also calls remote stays unmarked and rides on (someone else's
/// feed-apply surfaces it); replays merge `Unchanged` and never re-fire.
// [impl->REQ-NOTIF-1]
// [impl->REQ-PRES-1]
fn serve_notif_feed(brain: &mut Brain, stream_id: u64, origin: &str) -> DispatchOutcome {
    let store = match NotifStore::open() {
        Ok(s) => s,
        Err(e) => return DispatchOutcome::Failed(e.to_string()),
    };
    let policy = NotifPolicy::load();
    // This serving node's identity for the presence resolution: the broker's
    // handshake identity (the same hex peers see in our registry rows).
    let self_hex = brain
        .net_status()
        .ok()
        .and_then(|s| s.node_id_hex)
        .unwrap_or_default();
    if let Err(e) = brain.net_stream_subscribe(stream_id, 0) {
        return DispatchOutcome::Failed(e.to_string());
    }
    let mut decoder = NotifDecoder::new();
    let mut applied = 0usize;
    loop {
        match brain.read_event() {
            Ok(BrokerEvent::NetStreamData {
                stream_id: sid,
                bytes,
                ..
            }) if sid == stream_id => {
                let records = decoder.push(&bytes);
                if records.is_empty() {
                    continue;
                }
                match apply_notif_feed(&store, origin, &records, &policy) {
                    Ok(verdicts) => {
                        applied += verdicts.len();
                        surface_fresh_rows(&store, &records, &verdicts, &self_hex);
                    }
                    Err(e) => return DispatchOutcome::Failed(e),
                }
            }
            Ok(BrokerEvent::NetStreamEof { stream_id: sid, .. }) if sid == stream_id => {
                return DispatchOutcome::Served(format!("notif feed: {applied} record(s)"));
            }
            Ok(BrokerEvent::Error { message }) => return DispatchOutcome::Failed(message),
            Ok(_) => {}
            Err(e) => return DispatchOutcome::Failed(e.to_string()),
        }
    }
}

/// The receiving half of the cross-node notif redirect (REQ-PRES-1): first-
/// fire every **freshly inserted** undismissed row from an applied feed
/// batch, under this node's own presence resolution. Best-effort and loud —
/// a failed surface costs a boundary-resurface delay, never the feed.
// [impl->REQ-PRES-1]
fn surface_fresh_rows(
    store: &NotifStore,
    records: &[spt_net::net::notif::NotifRecord],
    verdicts: &[crate::notifsync::NotifApplyVerdict],
    self_hex: &str,
) {
    use crate::notifsync::NotifApplyVerdict;
    use spt_store::notif::MergeOutcome;
    let fresh = records.iter().zip(verdicts).filter_map(|(rec, v)| {
        let spt_net::net::notif::NotifRecord::Row { row } = rec;
        (matches!(v, NotifApplyVerdict::Applied(MergeOutcome::Inserted)) && !row.dismissed)
            .then_some(row)
    });
    let policy = crate::notif::NotifSurfacePolicy::load();
    let regs = crate::presence::load_registry_snapshots(&crate::presence::registry_snapshot_dir());
    let owlery = perch::owlery_dir();
    for row in fresh {
        match crate::notif::first_fire_at(store, row, &policy, &owlery, self_hex, &regs, now_ms()) {
            Ok(_) => {}
            Err(e) => eprintln!("NOTIF_FEED_SURFACE_FAIL:{}: {e}", row.notif_id),
        }
    }
}

/// WAN message arm: decode `WanMessage` lines, funnel each through the
/// access-gate + op-id-dedup path, until the peer finishes.
// [impl->REQ-NET-1]
fn serve_wan_feed(
    brain: &mut Brain,
    stream_id: u64,
    origin: &str,
    paths: &DispatchPaths,
) -> DispatchOutcome {
    if let Err(e) = brain.net_stream_subscribe(stream_id, 0) {
        return DispatchOutcome::Failed(e.to_string());
    }
    let mut decoder = WanDecoder::new();
    let mut handled = 0usize;
    loop {
        match brain.read_event() {
            Ok(BrokerEvent::NetStreamData {
                stream_id: sid,
                bytes,
                ..
            }) if sid == stream_id => {
                for msg in decoder.push(&bytes) {
                    let _ = receive_wan(&msg, origin, &paths.owlery);
                    handled += 1;
                }
            }
            Ok(BrokerEvent::NetStreamEof { stream_id: sid, .. }) if sid == stream_id => {
                return DispatchOutcome::Served(format!("wan feed: {handled} record(s)"));
            }
            Ok(BrokerEvent::Error { message }) => return DispatchOutcome::Failed(message),
            Ok(_) => {}
            Err(e) => return DispatchOutcome::Failed(e.to_string()),
        }
    }
}

/// Spawn the dispatcher on its own thread against the canonical broker
/// socket — the daemon-boot entry ([`crate::daemon::Daemon::run`]).
pub fn spawn_dispatcher(
    broker_name: String,
    registry: Arc<RegistryHost>,
    stop: Arc<AtomicBool>,
) -> thread::JoinHandle<()> {
    thread::spawn(move || {
        let paths = DispatchPaths::production();
        if let Err(e) =
            run_dispatch_loop(&broker_name, &paths, registry, &stop, DEFAULT_DISPATCH_POLL)
        {
            eprintln!("DISPATCH_LOOP_FAIL: {e}");
        }
    })
}

#[cfg(test)]
mod tests {
    use super::*;
    use serde_json::json;

    // [unit->REQ-NET-1] the demux table: every family's first record
    // classifies correctly, including the `request` kind collision.
    #[test]
    fn classifies_every_family_by_first_line_shape() {
        let sync = json!({"kind": "request", "sync_id": "pull-1", "refs": [], "have_tips": {}});
        assert_eq!(classify_first_line(&sync), StreamFamily::Sync);

        let attach = json!({"kind": "request", "session_id": 3, "from_seq": 0});
        assert_eq!(classify_first_line(&attach), StreamFamily::Attach);

        // [unit->REQ-INST-6] the rest op's opener disambiguates by shape too.
        let rest =
            json!({"kind": "request", "endpoint": "ling", "event": "suspend", "op_id": "a:1"});
        assert_eq!(classify_first_line(&rest), StreamFamily::Rest);

        let update =
            json!({"kind": "query", "upd_id": "u1", "current_version": 5, "channel": "stable"});
        assert_eq!(classify_first_line(&update), StreamFamily::Update);

        let notif = json!({"kind": "row", "row": {}});
        assert_eq!(classify_first_line(&notif), StreamFamily::Notif);

        let fetch = json!({"kind": "fetch", "path": "a.txt", "xfer_id": "x1"});
        assert_eq!(classify_first_line(&fetch), StreamFamily::Xfer);
        let push = json!({"kind": "push", "path": "a.txt", "total": 4, "xfer_id": "x2"});
        assert_eq!(classify_first_line(&push), StreamFamily::Xfer);

        let wan = json!({"target": "ling", "from": "oak", "body": "hi", "op_id": "w1"});
        assert_eq!(classify_first_line(&wan), StreamFamily::WanMsg);

        let reg = json!({"subnet": "home", "endpoint_id": "ling",
            "instance": {"node": "n1", "status": "Active", "epoch": 4}});
        assert_eq!(classify_first_line(&reg), StreamFamily::Registry);
    }

    // [unit->REQ-NET-1] unclassifiable shapes are Unknown — dropped loud,
    // never routed to a gate-bearing serve loop on a guess.
    #[test]
    fn unknown_shapes_refuse_classification() {
        for v in [
            json!({"kind": "chunk", "sync_id": "s", "offset": 0, "data_b64": ""}), // mid-protocol, not an opener
            json!({"kind": "request"}), // request without any discriminant
            json!({"kind": "request", "endpoint": "ling"}), // rest-ish but no event
            json!({"kind": "nonsense"}),
            json!({"target": "ling"}), // wan-ish but no op_id
            json!({"subnet": "home"}), // registry-ish but no instance
            json!({"completely": "other"}),
            json!(42),
        ] {
            assert_eq!(classify_first_line(&v), StreamFamily::Unknown, "for {v}");
        }
    }
}


Architecture clear. Key D2 insight confirmed: dispatcher + peer pump already speak pure IPC via `Brain::cold_start` over `broker_socket_name()` — they're just *spawned* in the broker process today. First migration = move spawn site broker→brain, **no new verb needed**. Wrinkle: net-gating (consumers only run when net up; self-heal attaches net late).

Let me verify net_status on net-less broker + peer pump spawn + check for existing D2 plan.


760	        let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(&mut body));
761	        if stop.load(Ordering::Relaxed) {
762	            return; // a clean stop — whatever the body returned
763	        }
764	        let ran_for = started.elapsed();
765	        match outcome {
766	            // The body only returns Ok when stop is raised; reaching here
767	            // without it is a died-loop fact — same restart as an error.
768	            Ok(Ok(())) => eprintln!("PEER_PUMP_EXIT: loop returned without stop — restarting"),
769	            Ok(Err(e)) => eprintln!("PEER_PUMP_FAIL: {e}"),
770	            Err(panic) => {
771	                let msg = panic
772	                    .downcast_ref::<&str>()
773	                    .map(|s| s.to_string())
774	                    .or_else(|| panic.downcast_ref::<String>().cloned())
775	                    .unwrap_or_else(|| "<non-string panic>".to_string());
776	                eprintln!("PEER_PUMP_PANIC: {msg}");
777	            }
778	        }
779	        backoff = next_backoff(backoff, ran_for, base);
780	        eprintln!(
781	            "PEER_PUMP_RESTART: supervised restart in {}s",
782	            backoff.as_secs()
783	        );
784	        let mut slept = Duration::ZERO;
785	        while slept < backoff && !stop.load(Ordering::Relaxed) {
786	            thread::sleep(TICK.min(backoff - slept));
787	            slept += TICK;
788	        }
789	    }
790	}
791
792	/// Spawn the pump on its own thread with the production wiring — the
793	/// daemon-boot entry ([`crate::daemon::Daemon::run`]). The thread body is
794	/// the SUPERVISOR ([`supervise_pump`]), not the bare loop: the pump
795	/// restarts after a panic or error with capped backoff (M8 decision 23).
796	// [impl->REQ-DAEMON-5]
797	pub fn spawn_peer_pump(
798	    broker_name: String,
799	    registry: Arc<RegistryHost>,
800	    cfg: DaemonConfig,
801	    stop: Arc<AtomicBool>,
802	) -> thread::JoinHandle<()> {
803	    thread::spawn(move || {
804	        let paths = PumpPaths::production();
805	        supervise_pump(&stop, SUPERVISE_BACKOFF_BASE, || {
806	            run_peer_pump(
807	                &broker_name,
808	                Arc::clone(&registry),
809	                &paths,
810	                production_resolver(),
811	                PumpConfig::from_config(&cfg),
812	                PumpHooks::default(),
813	                &stop,
814	            )
815	        });
816	    })
817	}
818
819	#[cfg(test)]
820	mod tests {
821	    use super::*;
822
823	    // The pump's cadence gate must prime on the first tick AND never compute
824	    // an instant before now (the underflow panic that killed the peer pump on
825	    // a sub-24h-uptime Windows host). `due` takes an Option and only ever runs
826	    // `now.duration_since(past)` forward — there is no `Instant - Duration`
827	    // arithmetic left to underflow.
828	    // [unit->REQ-HAZARD-INSTANT-UNDERFLOW]
829	    #[test]
830	    fn first_tick_is_due_and_never_underflows() {
831	        let now = Instant::now();
832	        // A day-long cadence on a process that may have started milliseconds
833	        // ago: the never-run leg is due regardless, with no subtraction-from-
834	        // instant anywhere in the path.
835	        assert!(due(None, now, Duration::from_secs(86_400)));
836	        // A leg that just ran is not due until its cadence elapses.
837	        assert!(!due(Some(now), now, Duration::from_secs(86_400)));
838	        // And it becomes due once enough has elapsed.
839	        let earlier = now - Duration::from_millis(50);
840	        assert!(due(Some(earlier), now, Duration::from_millis(10)));
841	        assert!(!due(Some(earlier), now, Duration::from_millis(500)));
842	    }
843
844	    // [unit->REQ-MESH-4] the pump fires a seed rotation only once its window has
845	    // closed: a due subnet rotates (epoch bumps, prior seed retained one-deep)
846	    // and its pending entry clears; a not-yet-due subnet is untouched and stays
847	    // scheduled. This is the daemon-side half of the coalescing window.
848	    #[test]
849	    fn fire_due_rotations_rotates_due_and_leaves_pending() {

        let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(&mut body));
        if stop.load(Ordering::Relaxed) {
            return; // a clean stop — whatever the body returned
        }
        let ran_for = started.elapsed();
        match outcome {
            // The body only returns Ok when stop is raised; reaching here
            // without it is a died-loop fact — same restart as an error.
            Ok(Ok(())) => eprintln!("PEER_PUMP_EXIT: loop returned without stop — restarting"),
            Ok(Err(e)) => eprintln!("PEER_PUMP_FAIL: {e}"),
            Err(panic) => {
                let msg = panic
                    .downcast_ref::<&str>()
                    .map(|s| s.to_string())
                    .or_else(|| panic.downcast_ref::<String>().cloned())
                    .unwrap_or_else(|| "<non-string panic>".to_string());
                eprintln!("PEER_PUMP_PANIC: {msg}");
            }
        }
        backoff = next_backoff(backoff, ran_for, base);
        eprintln!(
            "PEER_PUMP_RESTART: supervised restart in {}s",
            backoff.as_secs()
        );
        let mut slept = Duration::ZERO;
        while slept < backoff && !stop.load(Ordering::Relaxed) {
            thread::sleep(TICK.min(backoff - slept));
            slept += TICK;
        }
    }
}

/// Spawn the pump on its own thread with the production wiring — the
/// daemon-boot entry ([`crate::daemon::Daemon::run`]). The thread body is
/// the SUPERVISOR ([`supervise_pump`]), not the bare loop: the pump
/// restarts after a panic or error with capped backoff (M8 decision 23).
// [impl->REQ-DAEMON-5]
pub fn spawn_peer_pump(
    broker_name: String,
    registry: Arc<RegistryHost>,
    cfg: DaemonConfig,
    stop: Arc<AtomicBool>,
) -> thread::JoinHandle<()> {
    thread::spawn(move || {
        let paths = PumpPaths::production();
        supervise_pump(&stop, SUPERVISE_BACKOFF_BASE, || {
            run_peer_pump(
                &broker_name,
                Arc::clone(&registry),
                &paths,
                production_resolver(),
                PumpConfig::from_config(&cfg),
                PumpHooks::default(),
                &stop,
            )
        });
    })
}

#[cfg(test)]
mod tests {
    use super::*;

    // The pump's cadence gate must prime on the first tick AND never compute
    // an instant before now (the underflow panic that killed the peer pump on
    // a sub-24h-uptime Windows host). `due` takes an Option and only ever runs
    // `now.duration_since(past)` forward — there is no `Instant - Duration`
    // arithmetic left to underflow.
    // [unit->REQ-HAZARD-INSTANT-UNDERFLOW]
    #[test]
    fn first_tick_is_due_and_never_underflows() {
        let now = Instant::now();
        // A day-long cadence on a process that may have started milliseconds
        // ago: the never-run leg is due regardless, with no subtraction-from-
        // instant anywhere in the path.
        assert!(due(None, now, Duration::from_secs(86_400)));
        // A leg that just ran is not due until its cadence elapses.
        assert!(!due(Some(now), now, Duration::from_secs(86_400)));
        // And it becomes due once enough has elapsed.
        let earlier = now - Duration::from_millis(50);
        assert!(due(Some(earlier), now, Duration::from_millis(10)));
        assert!(!due(Some(earlier), now, Duration::from_millis(500)));
    }

    // [unit->REQ-MESH-4] the pump fires a seed rotation only once its window has
    // closed: a due subnet rotates (epoch bumps, prior seed retained one-deep)
    // and its pending entry clears; a not-yet-due subnet is untouched and stays
    // scheduled. This is the daemon-side half of the coalescing window.
    #[test]
    fn fire_due_rotations_rotates_due_and_leaves_pending() {


50:    InputReq, KillReq, NetDialReq, NetPresenceSubscribeReq, NetSent, NetStatusReply,
54:    KIND_NET_PRESENCE_SUBSCRIBE, KIND_NET_SENT, KIND_NET_STATUS, KIND_NET_STATUS_REPLY,
322:                KIND_NET_STATUS => self.dispatch_net_status(&send),
563:    /// the broker has no network host (`enabled: false`) so a brain can probe
568:            Some(host) => NetStatusReply {
569:                enabled: true,
574:            None => NetStatusReply {
575:                enabled: false,
582:            KIND_NET_STATUS_REPLY,
583:            serde_json::to_value(reply).expect("NetStatusReply serializes"),


No files found


`net_status` → `enabled: false` cleanly on net-less broker, and carries `node_id_hex`. The brain can poll it to gate consumers + get node_hex (no new verb). Let me read the net_status reply shape + peer pump's node_hex need.


558	            .resize(SurfaceSize::new(req.rows, req.cols))
559	            .map_err(|e| format!("resize failed: {e}"))
560	    }
561
562	    /// Report the broker-owned net endpoint's status (D4a). Answered even when
563	    /// the broker has no network host (`enabled: false`) so a brain can probe
564	    /// capability without treating absence as an error.
565	    // [impl->REQ-DAEMON-2]
566	    fn dispatch_net_status(&self, send: &SharedSend) {
567	        let reply = match self.net.get() {
568	            Some(host) => NetStatusReply {
569	                enabled: true,
570	                node_id_hex: Some(host.node_id_hex()),
571	                addr: host.addr_json(),
572	                conn_count: host.conn_count(),
573	            },
574	            None => NetStatusReply {
575	                enabled: false,
576	                node_id_hex: None,
577	                addr: serde_json::Value::Null,
578	                conn_count: 0,
579	            },
580	        };
581	        let frame = Envelope::new(
582	            KIND_NET_STATUS_REPLY,
583	            serde_json::to_value(reply).expect("NetStatusReply serializes"),
584	        );
585	        send_frame(send, &frame);
586	    }
587
588	    /// Dial a peer over the broker-owned endpoint (D4a).
589	    ///
590	    /// With an `op_id` the dial is routed through the broker-owned
591	    /// [`EffectJournal`] under the reserved [`NET_EFFECT_SESSION`] namespace,
592	    /// exactly like an idempotent PTY write: a replay of the same id after a

            .resize(SurfaceSize::new(req.rows, req.cols))
            .map_err(|e| format!("resize failed: {e}"))
    }

    /// Report the broker-owned net endpoint's status (D4a). Answered even when
    /// the broker has no network host (`enabled: false`) so a brain can probe
    /// capability without treating absence as an error.
    // [impl->REQ-DAEMON-2]
    fn dispatch_net_status(&self, send: &SharedSend) {
        let reply = match self.net.get() {
            Some(host) => NetStatusReply {
                enabled: true,
                node_id_hex: Some(host.node_id_hex()),
                addr: host.addr_json(),
                conn_count: host.conn_count(),
            },
            None => NetStatusReply {
                enabled: false,
                node_id_hex: None,
                addr: serde_json::Value::Null,
                conn_count: 0,
            },
        };
        let frame = Envelope::new(
            KIND_NET_STATUS_REPLY,
            serde_json::to_value(reply).expect("NetStatusReply serializes"),
        );
        send_frame(send, &frame);
    }

    /// Dial a peer over the broker-owned endpoint (D4a).
    ///
    /// With an `op_id` the dial is routed through the broker-owned
    /// [`EffectJournal`] under the reserved [`NET_EFFECT_SESSION`] namespace,
    /// exactly like an idempotent PTY write: a replay of the same id after a


Grep

node_id_hex|pub struct NetStatusReply|pub enabled|pub node_id_hex|pub addr


232-/// older broker) still produces/decodes a coherent reply (2.3 forward-compat).
233-#[derive(Debug, Clone, Serialize, Deserialize)]
234:pub struct NetStatusReply {
235-    /// Whether this broker hosts a net endpoint at all.
236:    pub enabled: bool,
237-    /// The node's stable name (lowercase-hex Ed25519 pubkey == iroh EndpointId).
238-    #[serde(default, skip_serializing_if = "Option::is_none")]
239:    pub node_id_hex: Option<String>,
240-    /// The endpoint's dialable address — a serde-serialized iroh `EndpointAddr`,
241-    /// opaque to this catalog (the net host owns its shape). `null` when disabled.
242-    #[serde(default)]
243:    pub addr: serde_json::Value,
244-    /// Live connections (accepted + dialed) in the broker's conn table.
245-    #[serde(default)]
--
252-    /// The peer's dialable address — a serde-serialized iroh `EndpointAddr`
253-    /// (opaque JSON here; the broker's net host decodes it).
254:    pub addr: serde_json::Value,
255-    /// The durable op id for exactly-once dialing. `Some(id)` routes the dial
256-    \ through the broker's [`crate::effect::EffectJournal`] (keyed under the


Everything confirmed. The first D2 migration is clean: **net consumers (dispatcher + peer pump) → brain child**, gated on `net_status().enabled` (existing verb, no new verb), broker keeps net bring-up + retry but drops the consumer spawn. Brain hosting them is strictly better — a broker restart no longer kills consumers, and the brain re-spawns them on respawn.

Let me set up tasks + check the REQ toml state.


ToolSearch

Question: select:TaskCreate


Question: select:TaskCreate


4-# KNOWN-HAZARDS invariant lives here as a REQ-* id (see docs/TRACEABILITY.md
5-# for the development contract).
6-#
7:# ACTIVATION MODEL: every requirement starts inactive (`required_stages = []`)
8-# so `check` stays green pre-code. A milestone (M0-M5) ACTIVATES its requirements
9:# by setting their real `required_stages`. Validate this seed against the
10-# installed CLI at M0 (the binary isn't installed yet).
11-
12-[scan]
--
16-
17-[policy]
18-# Default policy applied at activation: doc + impl + unit. Networking/lifecycle
19:# reqs add "int". Until a req is activated, its per-req `required_stages = []`
20-# keeps it out of the coverage gate.
21:required_stages = []
22-
23-# ───────────────────────────── Architecture / workspace ─────────────────────
24-[[requirements]]
25-id = "REQ-ARCH-1"
26-title = "Many small acyclically-layered crates"
27:required_stages = ["impl"]  # activated M0: spt-proto<-spt-store one-way layering (T14). impl-only: architecture is build-evidenced (cargo rejects cycles); grows as crates are added M1+.
28-[[requirements]]
29-id = "REQ-ARCH-2"
30-title = "Public SDK surface is spt-proto, spt-runtime, spt-msg"
31:required_stages = ["impl"]
32-[[requirements]]
33-id = "REQ-ARCH-3"
34-title = "Wire-protocol version independent of crate semver, N-1 compat window"
35:required_stages = ["impl", "unit"]  # activated M0: spt-proto wire version + compat window (T6)
36-[[requirements]]
37-id = "REQ-ARCH-4"
38-title = "Copy-verbatim the commodity layer from the sister project"
39:required_stages = ["impl", "unit"]  # activated M0: spt-proto EVENT grammar (T2)
40-
41-# ───────────────────────────── Daemon ───────────────────────────────────────
42-[[requirements]]
43-id = "REQ-DAEMON-1"
44-title = "One per-machine spt-daemon owning all per-machine state"
45:required_stages = ["impl", "unit", "int"]  # int activated M3b-B9: the daemon E2E (spawn→Psyche-loop→commune→brain-restart-survives→graceful-signoff) drives the consolidated daemon-hosted lifecycle through the real IPC
46-[[requirements]]
47-id = "REQ-DAEMON-2"
48-title = "Broker/brain split for seamless self-update"
49:required_stages = ["impl", "unit", "int"]  # int activated M3b-B9: the daemon E2E proves a brain kill+restart leaves the broker's PTY child + output stream intact and gapless (cross-cut with handoff.rs)
50-[[requirements]]
51-id = "REQ-DAEMON-3"
52-title = "Any api invocation auto-starts the daemon if absent"
53:required_stages = ["impl", "unit", "int"]
54-[[requirements]]
55-id = "REQ-DAEMON-4"
56-title = "Honor every KNOWN-HAZARDS invariant"
57:required_stages = ["impl", "unit", "int"]  # int activated M3b-B9: the daemon E2E proves graceful signoff runs the echo-before-teardown ordering (3.3) and removes the Self's ready marker in the consolidated daemon
58-
59-# ───────────────────────────── Storage ──────────────────────────────────────
60-# git-KV BranchStore substrate (ADR-0011, 2026-06-02; implement M4, optional M3c)
61-[[requirements]]
62-id = "REQ-STORE-1"
63-title = "spt-store::BranchStore (git branch as versioned KV; commit=checkpoint/tip=resume, atomic multi-key, merge-native sync) is the substrate for coarse/durable/audited state (context, registry snapshot+distribution, daemon checkpoint); hot paths (B5 fsync journal) + indexed queries (SQLite spool) excluded (ADR-0011)"
64:[Omitted long matching line]
65-
66-# ───────────────────────────── Harness contract — manifest seams ────────────
67-[[requirements]]
68-id = "REQ-MANIFEST-1"
69-title = "Per-adapter manifest with adapter_name and min_spt_core_version"
70:required_stages = ["doc", "impl", "unit"]
71-[[requirements]]
72-id = "REQ-SEAM-SPAWN"
73-title = "spawn-session seam"
74:required_stages = ["impl", "unit"]
75-[[requirements]]
76-id = "REQ-SEAM-POSTSPAWN"
77-title = "post-spawn / api bind seam with boot nonce"
78:required_stages = ["impl", "unit"]
79-[[requirements]]
80-id = "REQ-SEAM-PSYCHE"
81-title = "spawn-psyche seam (fresh + resume templates)"
82:required_stages = ["impl", "unit", "int"]
83-[[requirements]]
84-id = "REQ-SEAM-HISTORY"
85-title = "History subsystem (fetcher / locate-normalize / native store)"
86:required_stages = ["impl", "unit", "int"]
87-[[requirements]]
88-id = "REQ-SEAM-ACTIVITY"
89-title = "Activity/idle reported via api sentinels, not PTY quiescence"
90:required_stages = ["impl", "unit"]
91-[[requirements]]
92-id = "REQ-SEAM-INJECT"
93-title = "inject-input methods configurable per activity-state"
94:required_stages = ["impl", "unit"]
95-[[requirements]]
96-id = "REQ-SEAM-RESUME"
97-title = "resume-session seam (fresh-with-preload / continue-existing)"
98:required_stages = ["impl", "unit"]
99-[[requirements]]
100-id = "REQ-SEAM-CAPABILITY"
101-title = "Hostable endpoint-types capability declaration"
102:required_stages = ["impl", "unit"]
103-[[requirements]]
104-id = "REQ-SEAM-UPDATE"
105-title = "Adapter-update avenue (file-pull / delegated command)"
106:required_stages = ["impl", "unit"]  # activated M3c-C3: manifest [update] gains signing_key (file_pull content key) + self_verifies (delegated attest); adapter_update::plan_adapter_update dispatches each avenue (file_pull verified against the adapter key, delegated delegated-or-skipped)
107-
108-# ───────────────────────────── Harness contract — api surface ───────────────
109-[[requirements]]
110-id = "REQ-API-1"
111-title = "api prefix and adapter_name on every machinery invocation"
112:required_stages = ["impl", "unit", "int"]
113-[[requirements]]
114-id = "REQ-API-2"
115-title = "The api subcommand surface (bind/listen/poll/state/worker/boundary/...)"
116:required_stages = ["impl", "unit", "int"]
117-[[requirements]]
118-id = "REQ-API-3"
119-title = "commune/signoff are file-drops, not commands"
120:required_stages = ["impl", "unit", "int"]
121-
122-# ───────────────────────────── Startup flows ────────────────────────────────
123-[[requirements]]
124-id = "REQ-START-1"
125-title = "Adapters never resolve SPT_HOME; binary on PATH; api bridging only"
126:required_stages = ["impl", "unit"]
127-[[requirements]]
128-id = "REQ-START-2"
129-title = "Harness-hosted startup: api seed then listen"
130:required_stages = ["impl", "unit", "int"]
131-[[requirements]]
132-id = "REQ-START-3"
133-title = "spt-hosted startup: spawn-session then api bind (no file)"
134:required_stages = ["impl", "unit", "int"]
135-[[requirements]]
136-id = "REQ-START-4"
137-title = "Adapter-injected env aliases (SPT/OWL/LIVE)"
138:required_stages = ["impl", "unit"]
139-
140-# ───────────────────────────── Endpoints / Shells ───────────────────────────
141-[[requirements]]
142-id = "REQ-EP-1"
143-title = "Day-one endpoint types; open type system"
144:required_stages = ["impl", "unit"]  # activated M0: spt-proto endpoint taxonomy (T4)
145-[[requirements]]
146-id = "REQ-EP-2"
147-title = "Agent endpoints vs Shells distinction in the type model"
148:required_stages = ["impl", "unit"]  # activated M0: spt-proto endpoint taxonomy (T4)
149-[[requirements]]
150-id = "REQ-EP-3"
151-title = "Messaging payloads carry typed operation commands + file blobs"
152:required_stages = ["impl", "unit"]  # activated M0: spt-proto payload model (T7)
153-[[requirements]]
154-id = "REQ-EP-4"
155-title = "PresenceChannel broker endpoint (seam day-one)"
156:[Omitted long matching line]
157-[[requirements]]
158-id = "REQ-EP-5"
159-title = "Concrete shell instantiation model: spawn-mints-instance (vs relink/online), registered-on-node permission + broadcast-is-discovery, per-shell require_approval gate, max_instances_per_owner + over_cap, instance aliasing, discovery scope"
160:required_stages = ["impl", "unit", "int"]  # activated M5-D3a; int activated M5-D9b: the D3e shell E2E + the real-shell E2E (notify_shell_e2e.rs, CI-hook-driven against the standalone spt-shell-notify adapter) carry the int tags
161-
162-# ───────────────────────────── Instances ────────────────────────────────────
163-[[requirements]]
164-id = "REQ-INST-1"
165-title = "endpoint ID vs instance split (adapter-agnostic ID)"
166:required_stages = []  # rule 5: the data-model split shipped across M4 D3 (registry Instance rows) + D9-5 (home/adapter on the endpoint record); this umbrella req activates when instantiate-anywhere makes the split user-facing (instance minting) — which DEFERS PAST M5 (user decision 2026-06-04, M5-PLAN scope decision 2: consent framework seam only); its concrete slices are tracked by REQ-INST-7..15
167-[[requirements]]
168-id = "REQ-INST-2"
169-title = "Per-node files, synced Psyche mind"
170:[Omitted long matching line]
171-[[requirements]]
172-id = "REQ-INST-3"
173-title = "Dormant (warm) / suspended (cold) resting states"
174:[Omitted long matching line]
175-[[requirements]]
176-id = "REQ-INST-4"
177-title = "active to dormant/suspended fires a transition echo commune"
178:[Omitted long matching line]
179-[[requirements]]
180-id = "REQ-INST-5"
181-title = "Two-tier context sync (live to all, project to same-project)"
182:[Omitted long matching line]
183-[[requirements]]
184-id = "REQ-INST-6"
185-title = "Deferred messages not delivered to dormant/suspended instances"
186:[Omitted long matching line]
187-[[requirements]]
188-id = "REQ-INST-7"
189-title = "Subnet registry + bare-id resolution policy"
190:[Omitted long matching line]
191-[[requirements]]
192-id = "REQ-INST-8"
193-title = "Remote-control mode distinct from local operation"
194:[Omitted long matching line]
195-# ── Multi-subnet membership model (ADR-0006, 2026-06-01; activate M4) ──
196-[[requirements]]
197-id = "REQ-INST-9"
198-title = "Multi-subnet membership (same-user N subnets; cross-user seam)"
199:[Omitted long matching line]
200-[[requirements]]
201-id = "REQ-INST-10"
202-title = "Qualified addressing [subnet:]id[@node] + ambiguity forces qualification"
203:[Omitted long matching line]
204-[[requirements]]
205-id = "REQ-INST-11"
206-title = "spt rename <id> rippled to all instances (collision-checked, 6.5-reconciled)"
207:[Omitted long matching line]
208-[[requirements]]
209-id = "REQ-INST-12"
210-title = "Endpoint visibility per-(endpoint,subnet): excluded semantics, OR-of-defaults + override, gates sync"
211:[Omitted long matching line]
212-[[requirements]]
213-id = "REQ-INST-13"
214-title = "Subnet-exclusive sync + per-endpoint subnet-membership list"
215:[Omitted long matching line]
216-[[requirements]]
217-id = "REQ-INST-14"
218-title = "Resource advertisement (subnet resource registry): free-text blurb, both-authored, registry projection, visibility/whitelist-gated"
219:[Omitted long matching line]
220-[[requirements]]
221-id = "REQ-INST-15"
222-title = "Immutable home subnet (assigned at creation: auto-if-one/ask-if-many) + spt fork (cross-subnet clone to a new identity, copy-then-diverge, not re-home); adapter chosen at creation from registered hostable adapters, changed only via launch/resume-under-new (ADR-0010)"
223:[Omitted long matching line]
224-[[requirements]]
225-id = "REQ-REACH-1"
226-title = "Off-node remote-drive detection + file transfer"
227:required_stages = ["impl", "unit", "int"]  # int activated M4-D9-6: two-host rig E2E green (tests/twohost.rs — file fetch off B + push back to B across the rig; chunked xfer through the dispatcher gate)
228-[[requirements]]
229-id = "REQ-REACH-2"
230-title = "Remote command execution (deferred, consent-gated)"
231:required_stages = []  # rule 5: deferred by design (PRD: consent-gated remote exec is the highest-risk capability; workarounds exist) — M5+ with the instantiate-anywhere consent model
232-
233-# ───────────────────────────── Local messaging (M1) ─────────────────────────
234-# No PRD R-* covers local message delivery (foundational/implicit). Registered
--
237-[[requirements]]
238-id = "REQ-MSG-1"
239-title = "Local message delivery: TCP-first to a registered address, spool fallback when offline; id->address via registry (stale-clean first); reply routing (__REPLY_TO__)"
240:required_stages = ["impl", "unit", "int"]  # activated M1 T1 (impl+unit); int added T9: killer-quickstart cross-perch E2E
241-[[requirements]]
242-id = "REQ-MSG-2"
243-title = "spt binary CLI surface: send/ring/ready(+--once)/list/stop/whoami, stable arg shapes + exit codes"
244:required_stages = ["impl", "unit"]  # activated M1 T6-T8: spt binary, clap CLI, all 7 subcommands wired. M7 amendment (plan decision 2, 2026-06-05): `poll` removed — it duplicated `ready` (same run_listen dispatch); `ready --once` absorbs the drain-then-exit semantics. `spt api poll`/`api worker-poll` (machinery namespace) are unaffected. Pre-1.0, zero users, no deprecation shim.
245-[[requirements]]
246-id = "REQ-MSG-3"
247-title = "Ready-agent lifecycle: register perch (info.json + listener + registry address) on ready, drain spooled backlog on startup, clean teardown"
248:required_stages = ["impl", "unit", "int"]  # activated M1 T3 (impl+unit); int added T9: killer-quickstart backlog-drain E2E
249-[[requirements]]
250-id = "REQ-MSG-4"
251-title = "Listener stream stdout emits EVENT envelope lines (sister-format, ADR-0001): parse the __REPLY_TO__ frame, pass pre-formed typed envelopes through verbatim (no double-wrap), compose <EVENT type=\"msg\" from=…> otherwise, chunk oversized lines into EVENT-PART"
252:required_stages = ["impl", "unit", "int"]  # activated 2026-06-06 (envelope-missing fix): spt-msg::emit render seam + both listener loops (spt ready run_listen, spt api listen cmd_listen); int = quickstart E2E asserts the envelope shape on the live + backlog paths. Hook-channel drains (api poll / worker-poll) stay raw-frame by contract.
253-
254-# ───────────────────────────── Networking / pairing ─────────────────────────
255-[[requirements]]
256-id = "REQ-NODE-IDENTITY"

[Showing results with pagination = limit: 250]


Let me see the actual restoration REQ entries (they're at the end, the minted ones).


Bash

grep -n "BROKER-PROCESS-ISOLATION\|ROLLBACK-STATE-COMPAT" -A4 traceable-reqs.toml | tail -30

Show restoration REQ entries


638:id = "REQ-HAZARD-BROKER-PROCESS-ISOLATION"
639-title = "Broker and brain are separate processes: the broker runs as its own long-lived per-machine process that survives every brain restart, so a routine (brain-only) self-update restarts the brain onto the swapped binary while every hosted endpoint (PTY child, live QUIC conn, listening socket) stays untouched at the PROCESS level. The in-process-thread broker (daemon.rs:165-170) is a regression that silently unrealizes REQ-UPD-3 — apply degrades to an in-process Brain::handoff no-op and new code does not run until an unrelated restart (KNOWN-HAZARDS 6.7). Evidence must prove process-level survival (SPIKE-01/03 productionized as int: PTY child + live QUIC survive a brain-PROCESS restart onto a swapped binary), re-pointing the regression-masked in-process int tags currently on REQ-DAEMON-2 / REQ-UPD-3 (ADR-0018)."
640-required_stages = ["doc", "impl", "unit"]  # activated restoration D1 (ADR-0018 Q2/Q3): doc = KNOWN-HAZARDS 6.7 + ADR-0018 + design doc; impl = brainproc.rs (brain child entry + broker-side process supervisor) + daemon.rs spawn-brain-supervisor + the hidden `daemon brain` CLI entry; unit = brainproc supervisor respawn/backoff table + the real-binary brain_split.rs process-level smoke (broker survives a brain kill + respawns it). int activates at D7 = productionized SPIKE-01/03 process-level survival E2E (PTY child + live QUIC across a brain-PROCESS restart onto a swapped binary) + the new-brain x old-broker N-1 verb-surface compat test (V6), which also re-points the regression-masked in-process int tags on REQ-DAEMON-2 / REQ-UPD-3
641-[[requirements]]
642:id = "REQ-HAZARD-ROLLBACK-STATE-COMPAT"
643-title = "A brain must not irreversibly migrate durable state before update ready-promotion: the readiness-gated auto-rollback (ADR-0018 Q7) spawns the N-1 binary against durable state the new brain may have written, so every pre-ready write must stay N-1-readable (schema migrations gated behind ready-promotion, or written N-1-tolerant/additive). Else the first in-place schema migration silently bricks rollback (KNOWN-HAZARDS 6.8). Free now — a 2026-06-09 audit confirmed zero state-migration code exists; unmintable retroactively once a migration ships."
644-required_stages = ["doc"]  # doc activated restoration D1 (evidence: KNOWN-HAZARDS 6.8 + ADR-0018 + design doc V1). impl/unit activate at D6 (the readiness-gated auto-rollback): the N-1-readable pre-ready-write guard + two-phase applied record
645-# --- Added 2026-06-03 from grill-with-docs (ADR-0012, hazard 7.3); inactive until M4-D7.5 ---
646-[[requirements]]


REQ already `[doc,impl,unit]` active (int→D7). The migration commit adds more impl/unit evidence under the same req — no toml change. Setting up tasks.


TaskCreate

Write D2 plan + migration ledger

RESTORATION-D2-PLAN.md task breakdown + loop→verb→moves-at-commit ledger in RESTORATION-PLAN.md D2 section. Establish per-commit migration discipline.


Task #1 created successfully: Write D2 plan + migration ledger

Write D2 plan + migration ledger


Migrate net consumers (dispatch+peer pump) into brain

Move spawn_net_consumers from broker process to brain child, gated on net_status().enabled (existing verb, no new verb). Broker keeps net bring-up + net_retry_attach but drops consumer spawn. Brain polls net_status, spawns dispatcher+peer pump once net enabled. Pure once-only gate helper + unit test. First D2 commit.


Task #2 created successfully: Migrate net consumers (dispatch+peer pump) into brain


Run gates + commit net-consumer migration

cargo build/test/clippy + --no-default-features + traceable-reqs check + xtask check. Atomic commit with in-commit evidence tags. Push to dev-freeform branch for CI before any release.


Task #3 created successfully: Run gates + commit net-consumer migration


ToolSearch

Question: select:TaskUpdate


Question: select:TaskUpdate


Updated task #1 status


Now write the D2 plan doc + add the migration ledger to RESTORATION-PLAN.md D2.


Edit

- **Migration ledger (loop → verb → moves-at-commit).** Each commit moves
  exactly ONE consumer: define its versioned verb, reroute the direct call
  over the socket, move the loop into the brain child, prove the N-1 window.
  Keep the ledger current in-plan so "which loop runs where" is answerable at
  every mid-migration commit. **D2-close invariant:** every logic loop runs in
  the brain; the broker holds only continuity resources + the verb server.

- **Migration ledger (loop → verb → moves-at-commit).** Each commit moves
  exactly ONE consumer: define its versioned verb, reroute the direct call
  over the socket, move the loop into the brain child, prove the N-1 window.
  Keep the ledger current in-plan so "which loop runs where" is answerable at
  every mid-migration commit. **D2-close invariant:** every logic loop runs in
  the brain; the broker holds only continuity resources + the verb server.

  **Live ledger** (full task detail: `RESTORATION-D2-PLAN.md`):

  | # | Loop / consumer | Verb needed | Runs-where now | Status |
  |---|---|---|---|---|
  | D2-1 | Net consumers — inbound dispatcher (`dispatch.rs`) + outbound peer pump (`peerloop.rs`) | **none** — already pure IPC clients (`Brain::cold_start` over `broker_socket_name()`); gated on the existing `net-status` verb | **brain** ✅ | DONE |
  | D2-2 | Digest **parse** feed (`brain.run_digest_feed`) — the per-session projection that populates the hub | existing `subscribe`/output stream; the parse is already brain-side logic, only its *spawn site* (broker boot) moves | broker | TODO |
  | D2-3 | Shellwake reconcile (`shellwake::spawn_wake_host`) | **stays brain-owned** by exception (Q5): watcher children re-reconciled from disk on brain start | broker | TODO (move spawn) |
  | D2-4 | Psyche / pulse loops (`lifecycle.rs`) | per-loop verbs, additive (KH-2.3) | broker | TODO |
  | — | Net **bring-up** + `net_retry_attach` self-heal (REQ-DAEMON-9) | n/a | **broker** (stays — Q5, NetHost is broker-owned) | KEEP |
  | — | Digest **hub serve** (`digesthub::serve_digest_control`) | n/a | **broker** (stays — Q5, live projections + subscriber sockets) | KEEP |
  | — | Seed-control lock + liveness (`seedmap::serve_seed_control`) | n/a | **broker** (stays — Q2 anchor; D3) | KEEP |

  **D2-1 N-1 note (no-verb migration).** The consumers spoke pure IPC already;
  this commit only relocates their *spawn site* (broker process → brain child).
  No verb signature changed, so the verb-surface N-1 window is untouched. The
  one transient hazard — an old broker (pre-D2-1, spawns consumers in-process)
  paired with a new brain (also spawns them) → **double-spawn** — cannot occur:
  broker and brain are the same binary and swap together until D3 introduces
  brain-only swaps, by which point both halves are post-migration. Recorded so
  D3's first brain-only swap lands on an already-relocated consumer set.