# IR-21 instruments — status and limits Evidence behind the IR-21 remedy measurements in `docs/INFRA-REGISTER.md`. They live in the tree so a reviewer can re-run what the entry cites instead of taking its numbers on trust. Measured on `hfenduleam`, cargo 1.93.0, 2026-08-03. ## The instrument of record is the xtask port **Run `cargo run -p xtask -- binedge-check` and `cargo run -p xtask -- perjob-map`.** They are the maintained instruments: they ship with the workspace, build in CI, carry unit rows and a committed burn-down baseline, and are runnable by whoever is gating rather than only by an author with a python on PATH. They arrive with the xtask instruments lane. **The python scripts in this directory are retained deliberately, as the cross-validation specimens the port was diffed against.** That diff is part of the port's acceptance evidence — 185 site tuples identical, and 7 per-job rows identical, not merely four summary integers agreeing — so the reference arm has to stay reachable for the claim to stay checkable. Deleting them would orphan the evidence that the port measures the same tree the same way. Read them as specimens, not as the tool: a number quoted from this page is quotable because the port reproduces it, and the port is what a reader should run today. **Read the status column before quoting any number from these.** | instrument | status | what it establishes | |---|---|---| | `bindeps-probe/` | **COMPLETE** | artifact dependencies (remedy (a)) work, but only on nightly | | `binedge_check.py` | **SPECIMEN** (superseded by `xtask binedge-check`) | 185 sites, 146 guaranteed, 39 need an explicit build, 11 red | | `perjob_map.py` | **SPECIMEN** (superseded by `xtask perjob-map`) | 7 narrow invocations consume cross-package bins; **0 unguaranteed in CI** | ## `bindeps-probe/` — complete A two-crate workspace shaped like ours: `helper` declares a `[[bin]]`, `consumer` takes it as `artifact = "bin"` in dev-dependencies and resolves `CARGO_BIN_FILE_HELPER_helper_fixture`. Its own workspace root, so the parent workspace does not adopt it. Raw run logs are kept beside it — they are the evidence, not decoration: | log | arm | result | |---|---|---| | `out.txt` | stable 1.93.0 | **refused**, exit 101: ``artifact = … requires `-Z bindeps` `` | | `out2.txt` | nightly + `-Z bindeps` | **passes** — `test finds_the_helper_bin ... ok` | | `out3.txt` | nextest, `-Z` on the CLI | **fails**, exit 102 — nextest's inner `cargo metadata` never sees the flag | | `out4.txt` | nextest + `[unstable] bindeps = true` in `.cargo/config.toml` | **passes**, 1/1 | `out2.txt` is a positive control and it is the reason the stable refusal means anything: without it, exit 101 was equally well explained by a wrong scaffold or a wrong env-var name. Consequences recorded in the entry: golden runs nextest, so the **config** form is required — the CLI flag cannot be adopted per-invocation; and artifacts land under `target/debug/build///out/` as `CARGO_BIN_FILE_*`, not in `target//`, so adopting this rewrites the consumer path contract rather than swapping one expression for another. ## The population, completed 2026-08-03 — and the numbers it replaced The first version of these scripts detected **two** ways a test names a fixture bin and reported **33** needing an explicit build with **7** red. That was drawn from an incomplete population. `binedge_check.py` was corrected to detect **four**, and the figures were **re-derived and re-stated**, not patched upward. The port reproduces the corrected figures exactly, which is what makes the specimen worth keeping: | | two-syntax (superseded) | four-syntax (current) | |---|---|---| | consumer sites | 178 | **185** | | guaranteed (same-pkg integration) | 145 | **146** | | needs an explicit build | 33 | **39** | | red (needs one, has none) | 7 | **11** | The seven added sites, each accounted for rather than absorbed: four shared-resolver sites (`capture-player` ×3, `console-mode-probe` ×1 — all cross-package, all red), the same-package **unit** member `crates/spt/src/cli.rs` (needs a build, and has one at `ci.yml:106`), and two `spt`-binary sites (`spt-daemon/tests/twohost.rs`, `spt/tests/brain_respawn_rename.rs`). ## The per-job verdict — the 11 reds are NOT CI defects The per-job map uses the same four-syntax detector and answers the question the repo-wide allowlist could not: *does the job that RUNS a test build the bin it consumes?* ``` ci.yml unit nextest --workspace -E 'kind(lib)+kind(bin)' translate_proof_fixture PREBUILD golden.yml test nextest --workspace … translate_proof_fixture WORKSPACE golden.yml twohost-a cargo test -p spt-daemon --test twohost … spt WORKSPACE golden.yml twohost-a cargo test -p spt --test twohost_cli … mock-session WORKSPACE golden.yml twohost-b (same two) … WORKSPACE 7 narrow invocations consuming cross-package bins UNGUARANTEED: 0 ``` **Every one is guaranteed by its job**, either by an explicit prebuild or by an earlier workspace-shaped build that emits all 13 plain binaries. So the **11 reds do not become CI defects** — the repo-wide allowlist was over-strict in exactly the direction predicted, and a site with no explicit prebuild is still covered when its job builds the workspace. What the 11 remain is a hazard for **narrow invocations outside CI** — gate rigs and local runs, `-p`/`--test`/`--bins` against a cold pool — which is the population IR-21 says this class bites. The `ci.yml unit` row is the instrument's own validation case: under the two-syntax detector that lane was **invisible**, which produced a green over the one gap the class was filed for. It now appears and verdicts PREBUILD, which is what `ci.yml:105-106` actually does. ### Limits of the map, unfixed - Guarantee is modelled from `run:` blocks in workflow YAML. **Narrow cargo invocations also live inside PowerShell scripts** — `.github/ci/g6-curve.ps1:83` and `.github/ci/g6-postbounce.ps1:43` both run `cargo nextest run -p spt-daemon --test inject_control_wedge`, and no model that reads workflow YAML can know what a job built before invoking a script. **The port states this hole rather than leaving it to a footnote:** it scans those scripts with the same detector and reports the invocations as `UNMODELLED` with a measured count (2 today, of which 0 consume cross-package bins), so the limit is printed in the tool's own output next to the verdict. A verdict nobody can compute must not read green, and a limit that lives only in a doc is invisible to whoever runs the command. - `uses:` steps are not followed. - Guarantee is tracked per job in step order; it does not model artifact reuse across jobs on a persistent self-hosted workdir, which can mask a gap that a clean runner would expose. ### The four syntaxes `sibling_bin("")` and `CARGO_BIN_EXE_` were the original two. The others: - `crates/spt/src/cli.rs` — `current_exe()` → `deps/` → parent, then `format!("translate_proof_fixture{EXE_SUFFIX}")`. This is the unit-test member that `.github/workflows/ci.yml:105-106` hand-prebuilds for, i.e. the one case the class was filed over. It was invisible to the two-syntax detector — a green over the very gap the class exists for — and is now detected by both scripts. - `crates/spt-term/tests/support/fixture_bin.rs` — a shared resolver taking the bin name as a string, e.g. `fixture_bin("capture-player", "cargo build -p mock-adapter --bin capture-player")`. Its consumers (`capture-player` ×3, `console-mode-probe` ×1, in `spt-term/` and `spt-daemon/` tests) are all cross-package and all red — none has a prebuild anywhere in CI. They are four of the seven sites the completed population added. ### The over-match triage, completed A raw scan for bin names as string literals returns ~133 hits. **They are almost all non-consumers**, and the triage is now done rather than pending: - `mock-shell` (121) — an **adapter/shell-kind identifier**, not a path: manifest `name = "mock-shell"` (`spt-runtime/src/registry.rs`), `spawn_record(…, "mock-shell", …)` (`spt-daemon/src/shellhost.rs`), CLI arguments (`spt shell spawn mock-shell`). Only the `sibling_bin` uses resolve a binary, and those were already counted. - `mock-session` (6) — harness identifiers (`"mock-session".to_string()` in `harnesshost.rs`) and one manifest-content assertion. No path resolution. - `xlate_choreo_fixture` (1) — a `pgrep -f` **process-name** match in `inject_control_wedge.rs`, not a path. So the raw 133 contributed **zero** real consumers beyond those already found. The number was an over-matching upper bound, and reporting it as a site count would have inflated the class fourfold. Other known limits: block comments (`/* … */`) are not stripped, only line comments; and `perjob_map.py`'s guarantee predicate is per-job but its notion of "a workspace build emits every plain binary" is **false for kind-filtered runs** — `cargo nextest run --workspace -E 'kind(lib) + kind(bin)'` compiles harnesses and emits no plain fixture exe, which is exactly what `ci.yml:102-104` documents. That case is handled; it is recorded here because the first version of the script got it wrong and produced a green over the known-real gap. ## Running the instrument ``` cargo run -p xtask -- binedge-check # exit 1 if any consumer is unguaranteed cargo run -p xtask -- binedge-check --no-prebuilds # negative control: empties the allowlist cargo run -p xtask -- binedge-check --baseline # reds only on a site outside the burn-down set cargo run -p xtask -- binedge-check --sites # the full 185-row population, one line per site cargo run -p xtask -- perjob-map # per-job guarantee mapping ``` `--no-prebuilds` is the arm that matters: it must take the reds from 11 to 39 while the guaranteed sites stay green, and it must **not** reach the 11 same-package sites, whose guarantee comes from cargo rather than from any allowlist. A green without that arm carries no information. The port adds what a maintained instrument needs and a prototype did not have: a committed burn-down baseline (`--baseline`), a false-positive arm that **refuses** rather than reporting "0 wrongly flagged" once it can no longer recognize its own fixtures, and a derived `fix:` line on every red naming the exact `cargo build -p --bin ` that clears it. ## Re-running the specimens ``` python docs/instruments/ir21/binedge_check.py # the cross-validation reference arm python docs/instruments/ir21/binedge_check.py --no-prebuilds python docs/instruments/ir21/perjob_map.py ``` Use these to re-derive the cross-validation: diff `xtask binedge-check --sites` against `binedge_check.py`'s site listing and the two must agree row for row, on a tree whose `.rs` population is identical for both runs. Two populations wearing one measurement is the failure this comparison exists to exclude, so establish the tree first and the agreement second. Nothing here compiles the workspace — `cargo metadata --no-deps` plus a tracked-file scan. Worst observed wall time 1.41s for a script, of which `cargo metadata` is ~0.09s; the xtask commands run in ~2s warm, plus a build the first time in a cold pool.