# veilid — Repo Map (Rank 4 candidate, privacy-first)

## Brief refs: [^21][^22][^23][^24][^25][^26][^27][^44]

Source: `docs/research/inspiration/repos/veilid__veilid/` (GitLab mirror; license MPL-2.0; veilid-core v0.5.3).

## Workspace layout

`Cargo.toml:1-12` — workspace members:

- **veilid-tools** — runtime-agnostic primitives (tokio/async-std abstraction, async tag locks, IPC helpers). Embedded apps must depend on this transitively.
- **veilid-core** — node runtime: crypto, routing table, RPC, DHT, network manager, storage. THE crate for embedded integrators. `cdylib + staticlib + rlib`.
- **veilid-core/examples/basic** — minimal embed (`api_startup` → `attach` → run).
- **veilid-core/examples/private_route** — full app_message over private route, both sender + receiver.
- **veilid-server** — long-lived daemon (cli arg parsing, capnp client server). Skip for embedded SPT use.
- **veilid-cli** — TUI debug console talking to veilid-server over capnp. Skip.
- **veilid-flutter/rust**, **veilid-wasm**, **veilid-remote-api** — language bindings / browser. Skip.

## Integration points for SPT

### Embed veilid-core (no daemon)

- `veilid-core/src/lib.rs:70` — `pub use api_startup, api_startup_json, UpdateCallback`.
- `veilid-core/src/core_context.rs:267` — `pub async fn api_startup(update_callback: UpdateCallback, config: VeilidConfig) -> VeilidAPIResult<VeilidAPI>` — single entry. Returns a fully wired in-process node.
- `veilid-core/src/core_context.rs:238` — `api_startup_json(callback, json)` — same but config as JSON string.
- `veilid-core/src/veilid_api/api.rs:41` — `pub struct VeilidAPI` (Clone; cheap handle). Methods at `:59 shutdown`, `:142 get_state`, `:161 attach`, `:177 detach`, `:196 routing_context()`.
- Startup constraint (`core_context.rs:283-298`): only one live `(program_name, namespace)` pair per process; second `api_startup` returns `AlreadyInitialized`.
- `veilid-core/examples/basic/src/main.rs:44-82` — minimal embed pattern: build `VeilidConfig`, `VeilidTracing::stderr().try_apply_default_env()`, `api_startup`, `attach().await`, wait, `shutdown().await`.

### Identity (TypedKey, route IDs)

- `veilid-core/src/crypto/types/mod.rs:67-75` — `PublicKey`, `SecretKey`, `Signature`, `SharedSecret`, `HashDigest`, `NodeId`, `RouteId`, `MemberId` are all `impl_crypto_typed_and_group_and_vec!` — each is a `CryptoTyped<Bare…>` carrying a 4-byte `CryptoKind` (`VLD0`/etc) plus the raw bytes. Display form: `VLD0:<base64url>`.
- `veilid-core/src/crypto/types/mod.rs:78-79` — `KeyPair`, `RecordKey` (TypedGroup) — same FOURCC-prefixed encoding.
- `veilid-core/src/crypto/crypto_system/vld0/mod.rs:59` — `pub const CRYPTO_KIND_VLD0: CryptoKind = CryptoKind::new(*b"VLD0")` (Ed25519 + X25519 + ChaCha20-Poly1305 + BLAKE3 + Argon2). Only production crypto kind; `CRYPTO_KIND_NONE` is test-only.
- `veilid-core/src/crypto/mod.rs:213` — `Crypto::get(kind) -> Option<CryptoSystemGuard>` — fetch the cryptosystem to call `generate_keypair`, `sign`, `verify`, `decrypt_aead`, etc. Exposed via `api.crypto()` (`api.rs:89`).
- `veilid-core/src/veilid_api/types/route_blob.rs:5` — `pub struct RouteBlob { route_id: RouteId, blob: Vec<u8> }` — the shareable handle for inviting a peer. `blob` is the only thing you ship out-of-band.
- Node identity is auto-generated/persisted in the protected store on first start; not exposed as an explicit "generate identity" call. Observe `VeilidStateNetwork.node_ids` in the update callback (see `examples/basic` `:17-26`).

### DHT records

- `veilid-core/src/veilid_api/routing_context.rs:365` — `create_dht_record(schema: DHTSchema, owner: Option<KeyPair>, kind: Option<CryptoKind>) -> DHTRecordDescriptor` — owner-keyed record on the global DHT.
- `:403 open_dht_record(key, writer)`, `:435 close_dht_record`, `:486 delete_dht_record`, `:459 flush_dht_record`.
- `:510 get_dht_value(key, subkey, force_refresh)`, `:538 set_dht_value(key, subkey, data, writer)`, `:685 inspect_dht_record`.
- `:585 watch_dht_values(key, subkeys, expiration, count)`, `:623 cancel_dht_watch` — server pushes `VeilidUpdate::ValueChange` to the callback when peers update.
- Schemas: `veilid-core/src/veilid_api/types/dht/schema/mod.rs:13` — `pub enum DHTSchema { DFLT, SMPL }`. Constructors `:22 dflt(o_cnt)` (owner-only writes), `:25 smpl(o_cnt, members)` (multi-writer with per-member keys). `:48 max_subkey`, `:66 data_size`.
- `veilid-core/src/veilid_api/api.rs:211 get_dht_record_key`, `:234 generate_member_id`, `:251 transact_dht_records` (multi-record transactional updates via outbound_transaction_manager).
- Tuning: `veilid-core/src/veilid_config.rs:413-497` `VeilidConfigDHT` — `get_value_timeout_ms=10000`, `get_value_count=3`, `get_value_fanout=5`, `set_value_count=5`, `set_value_fanout=6`, `consensus_width=10`, `remote_max_records=128`, `remote_max_storage_space_mb=256` (defaults). All knobs are tunable per-application.

### App-level send/recv

Two RPC payload primitives — both use the routing context's safety selection.

- `veilid-core/src/veilid_api/routing_context.rs:332` — `app_message(target: Target, message: Vec<u8>) -> VeilidAPIResult<()>` — fire-and-forget. Up to **32768 bytes** payload.
- `veilid-core/src/veilid_api/routing_context.rs:266` — `app_call(target: Target, message: Vec<u8>) -> VeilidAPIResult<Vec<u8>>` — request/reply. Both directions capped at **32768 bytes**.
- `veilid-core/src/veilid_api/routing_context.rs:12` — `pub enum Target { NodeId(NodeId), RouteId(RouteId) }`. Without `footgun` feature, `Target::NodeId` is **rejected** (`:287, :346`) for `app_call`/`app_message` — receivers must accept only via private routes (sender anonymity by default).
- Receive side: in the `UpdateCallback` (`Arc<dyn Fn(VeilidUpdate) + Send + Sync>`) match `VeilidUpdate::AppMessage(Box<VeilidAppMessage>)` / `AppCall(Box<VeilidAppCall>)`. See `veilid-core/src/veilid_api/types/veilid_state.rs:301` for the full enum.
- `veilid-core/src/veilid_api/types/app_message_call.rs:8` — `VeilidAppMessage { sender: Option<NodeId>, route_id: Option<RouteId>, message: Vec<u8> }`. `sender = None` when received over a private route; `route_id = Some(_)` identifies which inbound private route delivered it.
- `veilid-core/src/veilid_api/types/app_message_call.rs:83` — `VeilidAppCall` adds a call-id used by `VeilidAPI::app_call_reply(call_id, message)` (`api.rs:414`) to send the response back through the route.
- Creating a private route to receive on: `api.rs:282 new_private_route() -> RouteBlob`, `:298 new_custom_private_route(crypto_kinds, stability, sequencing) -> RouteBlob`, `:373 import_remote_private_route(blob) -> RouteId`, `:389 release_private_route(route_id)`. Full bidirectional flow in `examples/private_route/src/main.rs:147-220`.

### Bootstrap config

Default seeds (the public Veilid network) are hard-coded in `veilid-core/src/veilid_config.rs:574-604` `VeilidConfigRoutingTable::default()`:

```
bootstrap: ["bootstrap-v1.veilid.net"]              // line 580 (native; wasm uses ws://… :5150/ws)
bootstrap_keys: [VLD0:Vj0lKDdUQXmQ5Ol1SZdlvXkBHUccBcQvGLN9vbLSI7k,
                 VLD0:QeQJorqbXtC7v3OlynCZ_W3m76wGNeB5NTF81ypqHAo,
                 VLD0:QNdcl-0OiFfYVj9331XVR6IqZ49NG-E18d5P7lwi4TA]
```

To run a **private network** (skip Veilid Foundation entirely):

- `veilid-core/src/veilid_config.rs:558` — `pub bootstrap: Vec<String>` — list of either TXT names (`bootstrap.your.tld`) or direct dial URLs (`tcp://1.2.3.4:5150`, `ws://…/ws`). Empty disables bootstrap (`routing_table/routing_domains/public_internet/controller/bootstrap.rs:22-25` early-returns).
- `veilid-core/src/veilid_config.rs:564` — `pub bootstrap_keys: Vec<PublicKey>` — V1 signed-bootstrap trust anchors. Set this to your own signing pubkey to refuse foundation bootstraps. Empty = V0 unsigned (unsafe; deprecated).
- `veilid-core/src/veilid_config.rs:625` — `pub network_key_password: Option<String>` — derives a network key that gates all RPC envelope authentication. Setting this on every node creates a fully partitioned overlay even if it shares dial info with the public net.
- `routing_table/routing_domains/public_internet/controller/bootstrap.rs:30-66` — bootstrap classifier: each string is parsed by `BootstrapDialInfoConverter::try_vec_from_url`; URLs → `network_manager.direct_bootstrap(di)`, plain hostnames → `txt_bootstrap(name)` (DNS TXT records).
- Operator-side setup steps: `BOOTSTRAP-SETUP.md:32-61` — generate signing keypair (`veilid-server --generate-key-pair VLD0`), put public half in every node's `bootstrap_keys`, point `bootstrap:` at your TXT-record host.
- For SPT: set `bootstrap = ["tcp://your-seed:5150"]`, `bootstrap_keys = [your_pubkey]`, `network_key_password = Some("…")`, and ship that config with the binary.

### Frame cap (64 KB)

Two distinct caps to remember — the "64 KB" frame is at the envelope layer, but app payloads are capped much lower:

- `veilid-core/src/crypto/envelope/mod.rs:12` — `pub const ENV0_MAX_ENVELOPE_SIZE: usize = 65507;` (max UDP-safe). Enforced at `:231` (parse), `:379` (decompress check), `:391` (assemble).
- `veilid-core/src/network_manager/mod.rs:60` — `pub const MAX_MESSAGE_SIZE: usize = ENV0_MAX_ENVELOPE_SIZE;` — single ceiling reused by every transport.
- Enforcement points per transport:
  - `network_manager/network/native/protocol/udp.rs:100,135` — UDP send + recv reject `>MAX_MESSAGE_SIZE`.
  - `network_manager/network/native/protocol/tcp.rs:50,87` — TCP framed read/write.
  - `network_manager/network/native/protocol/ws.rs:144,161` and `wasm/protocol/ws.rs:69,91` — WebSocket.
- App payload caps (much tighter, on top of envelope overhead):
  - `rpc_processor/coders/operations/operation_app_message.rs:3` — `const MAX_APP_MESSAGE_MESSAGE_LEN: usize = 32768;` (32 KiB).
  - `rpc_processor/coders/operations/operation_app_call.rs:3-4` — request and answer each `32768` bytes.
- **No built-in chunking/streaming**: anything larger than 32 KiB must be sharded by the application (or stored in the DHT and a pointer sent). The brief's "64 KB/RPC frame cap" is conservative for the envelope; the app-visible cap is 32 KiB.

### Pubkey identity model

- Every "id" type is `CryptoTyped<Bare…>`: 4-byte FOURCC (`b"VLD0"`) + raw key/hash. See `crypto/types/mod.rs:65-79`.
- VLD0 = Ed25519 sign/verify, X25519 ECDH, ChaCha20-Poly1305 AEAD, BLAKE3 hash, Argon2 password hashing (`crypto/crypto_system/vld0/mod.rs`). Provides `generate_keypair`, `sign`, `verify`, `cached_dh`, `decrypt_aead`, `encrypt_aead`, `hash`, `derive_shared_secret`.
- `NodeId` derived from the node's long-term Ed25519 public key (per crypto kind; a node holds keys for all enabled kinds — `VeilidConfigRoutingTable.public_keys/secret_keys`).
- `RouteId` is the public-key identifier of a private route's first hop; receivers publish it via `RouteBlob.blob` (an encrypted onion descriptor).
- `RecordKey` = owner Ed25519 pubkey (DHT records are owner-keyed).
- `MemberId` for SMPL DHT schemas — additional writer keys.
- All ids round-trip through human-readable form `VLD0:<base64url>` via `FromStr`/`Display` (used by the foundation bootstrap keys, `veilid_config.rs:585-589`).

### Bandwidth/CPU profile

Defaults that determine the steady-state cost (no live benchmarks captured; inferred from config + code shape):

- `VeilidConfigRoutingTable` (`veilid_config.rs:597-601`): `limit_over_attached=64`, `limit_fully_attached=32`, `limit_attached_strong=16` — node tries to keep ~16-32 live peers in its routing table at all times → continuous low-rate ping/pong traffic (`rpc_status.rs`) plus periodic `find_node` (`rpc_find_node.rs`).
- `VeilidConfigDHT.min_peer_count=20`, `min_peer_refresh_time_ms=60000` (`:481-482`) — refreshes peer set ~every 60s.
- `VeilidStateNetwork.bps_up/bps_down` (used in `examples/basic/src/main.rs:28-29`) is the public observable for runtime bandwidth.
- Per-send CPU: ChaCha20-Poly1305 AEAD + Ed25519 sign for every envelope (cheap, but every RPC including pings does it). DH cache (`crypto/dh_cache.rs`) amortizes the X25519 cost across messages to the same peer.
- Private route overhead: each `app_message` over a 2-hop safety route + N-hop private route does N+2 onion encryptions (chacha) and the message is re-enveloped at every hop. Default `default_route_hop_count` is set in `VeilidConfigRPC` (typically 1) — total ~3 hops, hence the brief's ~100-300ms latency figure.
- DHT operation cost is governed by `*_fanout` (5-6 parallel RPCs) + `*_count` (3-5 confirming responses) per get/set — every DHT op is dozens of envelopes.
- Storage caches dominate RAM: `VeilidConfigDHT` (`:484-489`) — `local_max_subkey_cache_memory_mb` defaults to `total_memory / 32` (capped at 256MB), `remote_max_storage_space_mb=256`. **Embedded apps should set these to <16MB** if not acting as a DHT replica.

## Examples worth copying

- `veilid-core/examples/basic/src/main.rs` — 83-line minimal embed. Copy the `VeilidConfigProtectedStore { always_use_insecure_storage: true, directory: … }` pattern only for prototypes (line 51 even has the warning).
- `veilid-core/examples/private_route/src/main.rs:147-220` — `create_route` and `open_route` show the full inbound/outbound private-route lifecycle: namespace separation per side (`config.namespace = "recv"|"send"`), `new_private_route()` + base64 blob exchange, `import_remote_private_route`, `Target::RouteId`-only sends, `try_again_loop` for `VeilidAPIError::TryAgain` startup race, `RouteChange` handling (`:234`).
- `veilid-core/src/tests/common/test_dht.rs:369` `test_set_dht_value_allow_offline` — DHT round-trip with the offline_subkey_writes path. `:518 test_all()` runs the suite.
- `veilid-core/src/tests/common/test_dht_transactions.rs` — multi-record DHT transactions (`api.transact_dht_records`).

## Binary size knobs

veilid-core's `Cargo.toml:21-94` features.

- **Default** = `["default-tokio"]` = `["enable-crypto-vld0", "rt-tokio"]`. Pulls tokio full features.
- **Swap runtime**: `default-features = false, features = ["rt-async-std", "enable-crypto-vld0"]` to use async-std. No "no-runtime" mode — pick one.
- `enable-crypto-vld0` (line 43-52) pulls `ed25519-dalek`, `x25519-dalek`, `curve25519-dalek`, `blake3`, `chacha20poly1305`, `chacha20`, `argon2`, `blake2`. Required for the real network. Disabling = no real crypto.
- `enable-crypto-none` (line 53-60) — test-only stub crypto. Cannot disable both.
- `enable-protocol-wss` (line 63) — only needed if your nodes talk WSS (TLS WebSocket). Add `webpki-roots`.
- `geolocation` (line 69) — pulls `maxminddb` + **`reqwest`** (huge: pulls hyper + http2). **Disable for embedded.**
- `footgun` (line 73) — re-enables `Target::NodeId` for `app_call`/`app_message` (bypasses sender privacy). Off by default. **Leave off.**
- `unstable-blockstore`, `unstable-tunnels` (line 76-77) — disabled in code (the API methods `find_block`, `start_tunnel` return `Unimplemented`). Skip.
- `virtual-network`, `virtual-network-server` (line 78-79) — simulation; skip for prod.
- `instrument`, `verbose-tracing`, `tracking`, `debug-locks` — all dev-only; skip.
- Hard non-optional deps (`Cargo.toml:99-129+`): `keyvaluedb=0.1.7` (sqlite-backed K/V), `rusqlite` (transitive via keyvaluedb), `directories`, `eyre`, `tracing`, `tracing-subscriber`, `clap`, `hashlink`, `weak-table`, `get-size2`, `range-set-blaze`. Substantial; veilid-core is not a small binary even minimal (>10 MB stripped is realistic).
- For SPT-style embedding: `default-features = false, features = ["rt-tokio", "enable-crypto-vld0"]` (or rt-async-std) — same as default minus the `geolocation` slot which is already off.

## Gotchas

- **One node per process per `(program_name, namespace)`** (`core_context.rs:283-298`). Multi-tenanting in a single process requires distinct namespaces and means duplicate state dirs on disk. Designed for "one app, one node".
- **Mandatory persistent storage**: `protected_store.directory` and `table_store.directory` must exist (or default platform paths used). veilid-core writes a sqlite DB + keyring entries even at startup. Not a pure in-memory mode.
- **Protected store on Windows**: `protected_store/native.rs` uses `keyring-manager` (Windows Credential Manager). If running under a service account without keyring access, set `always_use_insecure_storage: true` (insecure, dev only). Filesystem fallback at the configured directory.
- **WASM is a separate code path** with reduced caches (`veilid_config.rs:444-449` — `local_subkey_cache_size=128 vs 1024`, `remote_max_records=64 vs 128`). Also forces `ws://` bootstrap URL.
- **No NodeId-direct sends by default**: without `footgun`, you cannot `app_message` to a `Target::NodeId` (`routing_context.rs:287-289, 346-349`). Forces every conversation through a private route. SPT integration must always exchange `RouteBlob`s — there's no shortcut.
- **Bootstrap can fail silently to "no peers"**: `bootstrap_task_routine` (`routing_domains/public_internet/controller/bootstrap.rs:22-25`) early-returns on empty list. Combined with `VeilidAPIError::TryAgain` on every API call until the routing table is populated, integrators must wrap calls in retry loops (see `examples/private_route` `try_again_loop`).
- **Route death**: `VeilidUpdate::RouteChange` means the private route is dead — receiver must `new_private_route()` and re-distribute the blob (`examples/private_route/src/main.rs:234-238` note). DHT route auto-publish is roadmap (v0.6.0 per comment).
- **Startup sleep**: `examples/private_route/src/main.rs:31` includes a 10-second sleep before any veilid call — works around early-startup race conditions.
- **Bundled SQLite**: `keyvaluedb` brings in sqlite via `rusqlite` (with the `bundled` feature transitively). Same constraint SPT already has, but doubled.
- **MPL-2.0 license** — file-level copyleft. Modifying veilid-core sources triggers MPL obligations on those files; embedding the unmodified crate does not. Brief footnote [^44] flagged.
- **Geolocation feature is a trap door** for binary bloat (pulls `reqwest`); confirm `--no-default-features` builds disable it.

## Tests to study

- `veilid-core/src/tests/common/test_dht.rs:369 test_set_dht_value_allow_offline`, `:518 test_all` — DHT round-trips with a live API; shows config + assertion patterns.
- `veilid-core/src/tests/common/test_dht_transactions.rs` — `transact_dht_records` workflow.
- `veilid-core/src/tests/common/test_attachment_manager.rs` — full attach/detach lifecycle assertions.
- `veilid-core/src/rpc_processor/tests_rpc_processor/test_signed_value_data.rs` — signed DHT value invariants (good model for SPT envelope signatures).
- `veilid-core/src/rpc_processor/tests_rpc_processor/test_dial_info.rs` — dial info parsing (TXT/URL bootstrap forms).
- `veilid-core/src/veilid_api/tests_veilid_api/test_serialize_json/` — JSON wire format snapshots (relevant if SPT ever exposes a JSON control plane).
- `veilid-core/src/crypto/tests_crypto/test_crypto.rs` and `test_types.rs` — VLD0 sign/verify/AEAD/DH parity tests.
- `veilid-core/src/protected_store/tests_protected_store/` — keyring behavior; check Windows-specific assertions before depending on protected store in an installer.