--- name: free-space-floor-blocks-golden description: "hfenduleam's 32 GB CI floor killed 3 runs 2026-07-31, then FULL exhaustion (0 GB) 2026-08-01 surfacing as LNK1318 PDB errors; root cause = per-session scratchpad cargo targets (112 GB); reclaim recipe, by-path reap rail, measured before/after." metadata: node_type: memory type: project originSessionId: 19af1a2f-5196-41e5-8233-8bb5241f8280 modified: 2026-08-03T23:55:20.684Z --- The Windows CI jobs on `hfenduleam` assert a **32 GB free-space floor as step 2**, before any code runs. Below it the job fails with `RESOURCE=disk drive=C:\ free_bytes=… floor_bytes=34359738368`. **This reads as a red run but no test executed** — the sha is UNTESTED on that platform, not failing. Filed as `releases#56`. **Three casualties on 2026-07-31:** main's thin ci at `dd1653e` (27.8 GB), then BOTH Windows golden legs of run 30660216982 at `2a66d78` (15.5 GB), which took an otherwise 7-of-9-green v0.49.0 candidate to `conclusion=failure`. **I caused the golden one.** Running `cargo run -p xtask -- check` inside two throwaway worktrees left a **~7.3 GB target dir each** (14.6 GB), taking the box under the floor between pushing the golden ref and the Windows jobs starting. **Corrective: do not run `xtask check` in disposable worktrees on this host** — CI gates docs drift anyway, and running a local copy of the gate starved the real gate. If a local run is truly needed, point `CARGO_TARGET_DIR` at ONE reused scratch dir. **Reclaim recipe (14.5 GB → 51.1 GB in two steps):** 1. `git worktree remove` your finished worktrees — verify each is clean AND its head is pushed first. Plain `remove` (no `--force`) refuses a dirty tree, which is the safety you want. 2. Stale Claude session scratchpads under `%LOCALAPPDATA%\Temp\claude\C--Users-decid-Documents-projects-spt-core\\` hold redirected cargo targets — 5–14 GB each. **Delete only clearly-ended sessions (quote the mtime, not "age"), never one written today**: a live peer's build cache is recoverable but costs them a long rebuild, which is not yours to impose. Never touch the shared project `target/` (154 GB, in active use by other agents). **Reading a preflight red:** it is NOT a rerun-to-get-green situation — the platform never ran the code. Still hand it to the gater as a red per protocol; doyle's standing rule is no same-sha rerun without his triage ruling. Evidence the fix took: the `twohost` legs ran on the same box AFTER the reclaim and passed. See [[instrument-soundness-guards]] and [[v0480-published]]. **2026-08-01 — the class recurred at FULL exhaustion (0.00 GB free) and the root cause is now measured, not guessed.** It surfaces as `LNK1318: Unexpected PDB error` — sub-code `FILE_SYSTEM (3)`, then `LIMIT (12)` on retry — i.e. **disk exhaustion wearing a linker error's clothes**. Do not read the first LNK1318 as linker contention (I did, and was wrong); check `(Get-PSDrive C).Free` before diagnosing any link failure on this box. Where it went, measured: `spt-core/target` **156 GB**, `Temp\claude` **112 GB** (of which **111.5 GB was spt-core sessions alone**), `.worktrees/gate-target` 22.6 GB, `~/.cargo` 2.1 GB. **The 112 GB is not junk — it is per-session cargo target trees agents build inside their own scratchpads** (`scratchpad/tgt|target|target-merge/debug/`), ~7 GB each. The redirect instinct (avoid contending with the shared target) is RIGHT; the private-tree mechanism is WRONG. The fix is [[worktree-target-junction]] — junction the worktree's `target` onto ONE warm `.worktrees/gate-target`. Doyle called a fleet sweep (each agent drops its OWN scratch; ownership is not tellable from outside since all agents run as the same OS user) and is putting the junction in AGENTS.md. **Deleting the shared `target/` is doyle-authorizable, not unilateral** — it is pure cache but it is others' warm cache. His safety rail before deleting: sweep the process table for executables whose **path starts with** `spt-core\target` and reap **by path only, never by name**; a surviving handle makes the delete partially fail, so delete around it and note it. Result 2026-08-01: no holders, clean delete, **12.37 GB → 174.42 GB free** (162 GB reclaimed vs a 32 GB floor), gate-target warmth preserved. **Log free-GB before AND after — measured, not inferred.** The v0.50.0 golden then went green on ATTEMPT 1, the first single-attempt golden since this class started biting. **2026-08-03 — the class recurred at 0.04 GB, and it added a NEW DISGUISE plus a sizing trap.** doyle's LOCKSMITH respin gate: the **version bump invalidated the entire pool**, so a following `--workspace` test build wrote a SECOND generation of artifacts into `.worktrees/gate-ec5f38a/target` — **14.33 GB → 50.12 GB**. Reap of that subtree: free 0.04 → 47.60 GB, **47.56 reclaimed** (I re-measured independently: 47.62 GB free, target path gone). Classified before deleting per [[gate-worktree-target-disk.md]]: real dir not a reparse point, no inbound links, zero live cargo/rustc/link, own rig with verdicts already recorded. ⭐⭐ **THE SIZING TRAP: a version bump invalidates the whole pool, so the build AFTER a bump is not the size of the build before it.** Check free space before a cold full build; prefer package-scoped over `--workspace` on this box. This is the specific shape that ate the disk — not "the target grew". ⭐⭐ **THE NEW DISGUISE — a disk red wearing a NAMED TEST TARGET's clothes:** ``` LNK1180: insufficient disk space to complete link LNK1318: Unexpected PDB error; FILE_SYSTEM (3) error: could not compile (test "") due to 1 previous error ``` cargo reports it as a **compile failure of a specific named test**, which is exactly the sentence that invites "that test is broken." It is UNTESTED, not failed. **Never file it, never chase it, and never write it into FLAKE-LEDGER — a flake row for a disk red is a permanent lie about a test.** Re-run after reclaim and take the SECOND result as the first real one. If code was already changed in response to such a red, revert and re-derive: that was someone else's build artifact, not your defect. Same family as [[intermittent-green-is-zero-information]] — evidence that could have been produced by a cause unrelated to the subject is not evidence about the subject. **2026-08-03 golden 30860770146 @`b7b00c3` — ⭐⭐ THE FLOOR CAN BE UNSATISFIABLE BY CONSTRUCTION, and a cleanup-then-rerun green HIDES THAT.** `twohost-a` refused at step 2: `RESOURCE=disk drive=C:\ free_bytes=26224361472 floor_bytes=34359738368` (24.42 GiB vs 32). Proven untested by the STEP LIST, not by duration — steps 3–11 `skipped`, including `Run actions/checkout@v5` and both twohost ladder steps. Six legs green by NAME. Floor-gate readings across the one run: n1-gate start **47.6** → test start **84.8** (n1-gate released its pool) → twohost-a start **24.4**. The Windows test leg consumed **60.4 GiB and did not release it**, and `twohost-a` is `needs: test` on the SAME runner. So the box had 84.8 GiB free when the consumer started and still could not clear a 32 GiB floor after it: passing would require **~92 GiB free at test-leg start**. ⭐⭐ **A floor placed AFTER the biggest consumer on the same runner is not a dirty-box problem and is not fixed by starting cleaner.** One-off cleanup makes the next run green and leaves the mechanism intact — discount any green obtained that way. ⚖ **Verify the SOLE-OCCUPANCY premise before accepting a consumption attribution.** "The test leg consumed 60.4 GiB" needs "nothing else ran on that box in the window" — a different claim. Checked from job timings: n1-gate(Windows) cleared 23:07:14, test ran 23:07:16→23:48:14, twohost-a started 23:48:17. Sole occupancy, so the attribution held. See [[ruling-rests-on-a-premise]]. ⚠ `gh run rerun --failed` does NOT re-run the passing test leg, so the retried job starts against a box state **the full graph never produces**. The resulting green proves the RETRIED LEG at that sha; it does NOT prove the job graph fits the box. Two different-sized claims — record only the one bought. Same sha, so tested==shipped is untouched. ⚠ Compute the rerun set only once every job has CONCLUDED — `--failed` snapshots at invocation, so firing while a sibling is still in flight can miss or race it. **Second resource class, same day (attempt 2 at `2a66d78`):** floor PASSED (86.3 GB), then the Windows test leg died at BUILD time — `rustc-LLVM ERROR: out of memory` compiling spt-daemon lib test, cargo exit 101, no test binary. Same UNTESTED-not-failing reading. Jobs sequential, not contention. Host: 63 GB RAM / 15360 MB pagefile (undersized commit limit + parallel rustc codegen = the fitting shape, NOT measured at OOM — kept as shape, not finding). Doyle's rulings: ship- Windows-untested REJECTED; lever = runner-level machine-scope `CARGO_BUILD_JOBS=4` + service restart (SHA-PRESERVING — a workflow-file edit would mint a new commit and forfeit tested-sha == shipped-sha); pagefile resize = operator machine config, flagged needs-operator on #56. ## 2026-08-24 — the reap went fine; the two FIGURES handed to me were both wrong, in the same direction SIGNET #218 release window. doyle sequenced my `signet-w1` lane pool for reap and handed off **131G free / 112G pool**. deployah, driving the release, measured **77.2 GB free** himself and asked for the reap NOW rather than after (golden preflight floor 32 GB). I measured before touching anything: **76.43 GB free**, pool subtree **99.95 GB / 54,989 files** by a walk. Reap: **76.43 → 170.31 GB = 93.88 GB reclaimed** (lower than the pool's own size because `assembly-signet-218` was building through the window — say which, or the two numbers look like an error). deployah re-measured 169.65 GB independently. Classified teardown per [[gate-worktree-target-disk]]: real dir not a reparse point, `CARGO_TARGET_DIR` unset, inbound reparse sweep over every `.worktrees/*/target` + root target found none aimed at it, and the one live `cargo` resolved **by image path** to a DIFFERENT pool (`assembly-signet-218`) — that check is what made it safe, not the absence of cargo. Claim dropped first (`xtask pool-release`, exit 0), then the target SUBTREE only; worktree intact at `ad3347ad`. ⭐⭐ **A free-space figure is an INSTANT, not a property — and a hand-off strips the timestamp.** doyle's 131G was real when he took it (pre-assembly-clippy) and was relayed as if it were headroom. He accepted the correction and ruled the mechanism on himself: **future hand-offs carry free-space WITH its as-of, or defer to the preflight**, which measures at fire time. The system held here only because deployah's floor step re-measures — but a high figure in a hand-off is the wrong direction to err: it silently clears a floor it should not have, and IR-59 had already held one release that week. ⭐⭐ **A size from an INDEX is not a size from a WALK.** doyle's 112G was Everything's index; my `Get-ChildItem -Recurse | Measure-Object Length` walk said 99.95 GB. An indexed size is a cached claim wearing a measurement's clothes — same species as [[derived-view-vs-on-disk-record]]. Walk it before you quote it in a teardown or a floor decision; use the index to LOCATE, never to SIZE. **What actually settled it: two instruments, not one confident sample.** deployah's own words — "two instruments agreeing against doyle's 131G is what settles it, not my single sample." Neither deployah's 77.2 nor my 76.43 alone outranks a peer's confident figure; their agreement does. Inverse of [[identical-readings-across-opposite-outcomes-indict-the-meter]]: there, two agreeing readings indicted the meter because they came from the SAME idiom; here they corroborate because the idioms were independent. **Agreement is evidence only when the instruments differ.** ⭐⭐ **BOTH DIRECTIONS OF THE ATTRIBUTION ERROR, measured hours apart on 2026-08-24 — a free-space delta measures the BOX over an interval, not your action, unless you owned the interval.** Morning reap: pool walked at **99.95 GB**, free moved only **93.88 GB**, because `assembly-signet-218` was building through the window. Evening reap of the fixup arm pool: pool walked at **5.97 GB**, free moved **14.09 GB** (204.61 → 218.70) — a delta *larger* than the thing deleted, so ~8 GB came from something else releasing concurrently and none of it was mine to claim. **Report the WALK as your reclaim and the delta as the box's movement, and say which is which.** A delta that happens to match the pool size is the flattering case and is not evidence you owned the window either — that is the same sole-occupancy premise this file already demands for consumption attributions, read from the reclaim side. See [[an-unexercised-prediction-is-not-a-correct-one]] for the sibling habit of sorting what your evidence actually exercised.