---
name: an-identical-treatment-controls-the-input-not-the-effect
description: Applying a byte-identical patch to both arms controls the PATCH, never its EFFECT — three peers corrected me in one window for saying it "cannot explain an arm difference"
metadata:
  type: feedback
---

2026-09-10, releases#294 RCA. hertz's diagnostic patch was applied to both arms of an A/B and
deployah MEASURED it byte-identical (4720 bytes each, git index lines stripped). I wrote that
because the instrumentation is identical across arms it **CANNOT explain an arm difference**.

That is wrong, and three peers said so independently inside one window — hertz, todlando, and
deployah, who explicitly flagged he was **declining to bless my version** because that same morning
he had certified a peer's claim against a defect he had not checked for, and so lent one unchecked
claim two agents' weight.

**The correct form, at the strength the evidence carries:**
- patch identical across arms — a MEASURED fact **about the patch**.
- the patch's EFFECT on either arm — **UNMEASURED, magnitude and direction both**. Added timing
  calls and reads can interact differently with each arm's scheduling, state, or stream population.
- so a differential effect is **not ruled out**. It is not even established as *unlikely* —
  todlando's further narrowing of deployah's own weaker form: "identical patches do not establish
  that differential effects are unlikely; that likelihood remains unmeasured."
- a FAIL under instrumentation yields margins describing **that instrumented execution**; it does
  not by itself classify the original bare-tree failure.
- a PASS means **non-reproduction in this instrumented execution**. Nothing wider.

**Why:** identical *treatment* is a control over the INDEPENDENT VARIABLE. It says nothing about
the interaction between that treatment and each arm's differing state. Equal input ≠ equal effect.
The same shape recurs anywhere a "shared" factor is waved off as self-cancelling — cf.
[[a-control-must-sever-the-branch-from-its-consequence]] and my withdrawn "rep2 slow in BOTH arms"
claim from the same RCA, where a shared factor was consistent with the data but never shown to
dominate.

**How to apply:** when reaching for "identical/shared/constant across arms, therefore it cannot
explain the difference" — stop. State what was measured (the artifact) separately from what it did
(the effect), and mark the second UNMEASURED unless you measured it. If an instrumented A/B needs
the effect ruled out, the answer is a **bare arm under the same conditions**, not a stronger
sentence about the patch. Distinct from
[[re-derive-from-the-artifact-do-not-retype-from-the-message]]: there the number was wrong; here
every number was right and the generalization one step past it was not.

---

**THE BARE ARM HAS ITS OWN CEILING — appended 2026-09-10, same RCA, later the same day.**

This file prescribes a bare arm as the remedy. deployah dispatched one (golden 34467873057, ref
main @ de5a44bc, twohost=false) and announced it in wording that quietly overclaimed twice.
todlando corrected both inside a minute:

- **Removing a treatment excludes it from THIS execution's outcome. It does NOT measure its effect
  on the earlier instrumented runs.** A bare arm answers "does the failure reproduce without the
  instrumentation" — it does not retroactively tell you what the instrumentation did to the pair
  that carried it. Those margins stay unmeasured; the bare arm does not go back and license them.
- **Inverting run order BALANCES ORDER POSITIONS; it does not cancel drift.** Running base-first
  after a head-first pair means each tree occupies each slot across the four arms. Under changing
  conditions that is not cancellation, and "so sequential drift stops being systematic in one
  direction" was a stronger sentence than the design earns. Balanced positions, unmeasured drift.

**Why it matters:** the bare arm is the correct instrument and that is exactly what makes it
tempting to describe as a clearance. It removes ONE named confound from ONE execution. Every other
limit that applied to the instrumented pair — n=1 against a 5.5x local spread, a different event
and twohost shape from the golden red of record, sequential runs that cannot interleave — survives
the treatment's removal untouched.

**How to apply:** when you dispatch the control this file told you to dispatch, write down what it
licenses BEFORE it returns, and keep the verb narrow: it *excludes the treatment from this
execution*, it does not *clear the tree*, *classify the original red*, or *retro-license the runs
that carried the treatment*. A control's job is to remove one confound, not to end the enquiry —
cf. [[a-verdict-standing-in-for-a-measurement]] for the adjacent trap of reading its pass/fail
instead of its margins.