---
name: a-pr-open-is-a-ci-battery-on-both-runner-boxes
description: Opening a PR fires thin CI onto both self-hosted runner boxes at once; beside a builder battery it OOM-kills the kitsubito runner (15 GB) and the service does not self-recover — open PRs only when both boxes are free
metadata: 
  node_type: memory
  type: feedback
  originSessionId: 7024face-6a64-4294-8609-8fafebc8e4df
  modified: 2026-09-07T04:03:50.748Z
---

**Measured 2026-09-07 (W1 #249, PR #196 at 8d980fdf):** I told todlando "open the PR at the respin tip" and he did at 01:40Z, while his batteries ran on BOTH boxes. Thin CI run 34073766394 landed on both self-hosted runners within seconds: the Windows unit job died on "There is not enough space on the disk" (this box was at 0.5 GB free from his test-profile build), and on kitsubito the CI job's `rust-lld` was OOM-killed inside the runner service's cgroup (15 GB RAM, three OOM kills 01:48Z, journal `Killed process … (rust-lld)`, memcg = the runner unit) — the unit went `failed`, runsvc's "re-launch in 5 seconds" did NOT bring it back, and every later Linux CI job would have queued forever. Both reds carried ZERO test signal (0 Summary, 0 panics) and read as "the head is red" from the PR page.

**Why:** the "one battery per box" rule is about load on the runner boxes, and a PR open IS a battery on both of them — the golden boxes are the same machines the builders and I batter. A cold `nextest` battery plus a CI unit job's link step exceeds kitsubito's memory; the runner service is the process the kernel picks.

**How to apply:**
- A PR is opened only when BOTH boxes are free of batteries (builder's and mine), and the CI rerun after a fix goes in the box order like any other battery, never beside one. Say this in the dispatch, not just "open at the tip".
- After any CI job that ends "runner has received a shutdown signal" or a kitsubito red with no Summary: `ssh reavus@kitsubito systemctl is-active actions.runner.BigscreenVR-spt-bs-core.kitsubito.service` and read `journalctl -u` — an OOM-killed unit stays `failed`; `sudo systemctl reset-failed && start` (passwordless sudo works for reavus) restores it; confirm `Runner.Listener` by pgrep.
- Pull job logs from `gh api repos/O/R/actions/jobs/<id>/logs` and grep `Summary \[` / `panicked at` / `not enough space` / `shutdown signal` BEFORE reading any red as the head's.
- INFRA-REGISTER candidate: the kitsubito runner has no memory headroom for a concurrent battery; either a `MemoryMax`/nice for batteries or a CI concurrency gate keyed on the lane lock.

**Second face, measured 2026-09-07 04:00Z (W1 land):** the ff push to main fires the SAME full `ci` workflow (run 34081163501, `main@9489ef60`, `push`, created 03:53:25Z = one second after my `git push … :refs/heads/main`). I broadcast "cargo lifted, boxes free" at 03:54Z with that run one minute old on both boxes; hertz then claimed a cold pool here beside it. The post-land window is NOT free for ~20 min. Corollary: the Windows unit job's `actions/checkout@v5` (default `clean: true` = `git clean -ffdx`) deletes the previous job's ~50 GB runner target at `C:\actions-runner\_work\spt-bs-core\spt-bs-core\target` — that is the +52 GB free (75 → 127) I could not attribute to any reap. A free-space jump right after a CI run is the runner, not a peer's reap; and that space comes BACK during the next job, so read the floor right before a battery, never after a land.

See [[dispatch-window-legs-must-name-the-golden-box]], [[prefire-check-reads-the-disk-floor]], [[in-flight-run-logs-are-gated-read-steps-and-annotations]], [[failed-job-is-not-a-test-victim]].