1	# Bigscreen Beyond — Calibration & Distortion Pipeline
2
3	Sources (all first-party, read-only; branch `main` unless noted):
4
5	- `calibot` — C:\Users\decid\Documents\projects\calibot (factory tracking-calibration rig)
6	- `valve-tools-wrapper` — C:\Users\decid\Documents\projects\bigscreen\valve-tools-wrapper (Python wrapper around Valve factory tools + sample per-device configs)
7	- `big-notebooks` — C:\Users\decid\Documents\projects\bigscreen\big-notebooks (ops/fulfillment notebooks + config flasher)
8	- Supplementary (found adjacent, directly relevant): `vertical-alignment-proximity` (branch `master`) — C:\Users\decid\Documents\projects\bigscreen\vertical-alignment-proximity (per-unit optical alignment station, "VAP"); `check-canting` — C:\Users\decid\Documents\projects\check-canting; `extended-mode-display` — C:\Users\decid\Documents\projects\extended-mode-display (non-SteamVR display access).
9
10	---
11
12	## 1. The big picture
13
14	The Beyond is, to SteamVR, a **stock lighthouse HMD**. Its tracking SIP is a Tundra Labs module enumerating with Valve's VID/PID (`0x28de:0x2300`, see `calibot/calibrate_diode.py` `VIDS_TUNDRA/PIDS_TUNDRA`); a separate Bigscreen MCU ("Atmel", `0x35bd:0x0101`, product string "Beyond") handles displays/LED/proximity/fan. **All optical + tracking calibration lives in a single Valve-format JSON config stored in the tracking SIP's flash** (~2 KB zlib-compressed, ~11 KB inflated — see notebook output in `big-notebooks/testConfigFlasher.ipynb`: "Read config of 1979 bytes ... inflated to 11004 bytes"). SteamVR's built-in lighthouse driver downloads that JSON from the device at startup, caches it under `C:\Program Files (x86)\Steam\config\lighthouse\<lhr-serial>\`, and implements `IVRDisplayComponent::ComputeDistortion()` from its `tracking_to_eye_transform` block. The compositor samples that function to build its warp mesh. No Bigscreen-authored OpenVR driver exists in any of these repos — the distortion consumer is Valve's closed `driver_lighthouse`.
15
16	Factory chain per unit:
17
18	1. **calibot station** — flashes a full template config (incl. design-level distortion polynomials) to the SIP, then solves per-unit photodiode geometry with Valve's `lhcalib.exe` and writes it back to SIP flash.
19	2. **VAP station** — calibrates the proximity sensor (into the MCU's 512-byte TLV config memory) and measures per-unit display/optics rotational misalignment with a camera; writes per-unit `eye_to_head` matrices and `ipd.default_mm` into the SIP JSON config.
20	3. Results PASS/FAIL go to a factory MES web endpoint (`vertical-alignment-proximity/mes_logger.py`, station IDs "VAP T060, MMI T070, BT T080"). The Bigscreen cloud (`big-notebooks/api.py`) stores order/IPD/face-scan data, **not** optical calibration; no evidence of per-serial calibration in the cloud.
21
22	---
23
24	## 2. calibot — what it is
25
26	`calibot/README.md`: "Software run on calibot - used for steamVR tracking". It is a **2-axis motorized motion rig** (two Trinamic TMCM-1161 stepper modules over serial-TMCL, worm ratio 180:1, axes addressed by global-parameter IDs 159/160 — `calibot/calibot.py`). It slings up to **4 headsets at once** through a randomized motion path in view of **2–3 lighthouse base stations** (`calibot/calibot_config.yaml`: `basestation_count: 3`).
27
28	Flow (`calibot/calibrate_diode.py`, the shipping app; `calibot/main.py` is an older variant):
29
30	1. USB device-monitor matches each Atmel MCU to its Tundra SIP by shared USB hub port chain (`HeadsetHIDMatching`); headset LED is driven via MCU HID feature report `[0,'L',r,g,b]` for operator status (white=in progress, green=pass, red=fail).
31	2. `calibot/load_init_json.py` uploads `calibot/init_json_config.json` **wholesale** to the SIP via `lighthouse_console.exe uploadconfig`, with only `device_serial_number` substituted per device. I.e. at this stage **every unit receives identical distortion polynomials** — the optical distortion model is per-design (possibly per lens revision), not measured per unit.
32	3. Runs Valve's `lhcalib.exe`:
33	   `lhcalib /bodyserial <LHR-…> /bodycal init_json_config.json <total_hits=800> <hits_per_sensor=80> /bodycalmulti <n_bases> /deletemissingsensors`
34	   which solves photodiode positions/normals from sweep observations during arm motion. Output file `auto_<serial>.json` echoes the input config with solved `lighthouse_config` (`channelMap` [22 sensors], `modelPoints` [m, HMD frame], `modelNormals`).
35	4. The station downloads the device's current config, replaces only `lighthouse_config`, adds a `bscv` version key (`"<total_hits>.<hits_per_sensor>"`, e.g. `800.80`), and re-uploads (`calibrate_headset()` in `calibot/calibrate_diode.py`).
36	5. IMU calibration support exists (`ImuCalibrationProcess` → `acc_bias`, `acc_scale`, `gyro_bias`) but is commented out in `calibot/main.py::create_imu_calibration_threads` — production template ships zero biases.
37
38	So calibot measures: **photodiode geometry per unit** (tracking), optionally IMU bias. It does **not** measure optics.
39
40	---
41
42	## 3. The on-device config JSON (the calibration artifact)
43
44	Canonical template: `calibot/init_json_config.json` (production Beyond 1, `revision: 0.12`). Older: `calibot/init_json_config_o.json` (differs only in serial placeholder, `ipd.default_mm` 63 vs 73, and slightly different nominal `modelPoints`); `valve-tools-wrapper/steamvr/init_json_evt.json` (EVT era); 20 real DVT-unit dumps in `valve-tools-wrapper/steamvr/tools/bin/win64/DVTs/LHR-*.json`; a Beyond-2 test config embedded in `big-notebooks/testConfigFlasher.ipynb` (`BS2_TEST_CONFIG.json`, file itself not in repo).
45
46	Top-level keys (all variants):
47
48	| key | meaning / production value |
49	|---|---|
50	| `manufacturer`/`model_number` | "Bigscreen"/"Beyond" |
51	| `device` | `eye_target_width/height_in_pixels: 2544×2544` per eye, `first_eye eEYE_LEFT`, `num_windows: 1` |
52	| `direct_mode_edid_vid/pid` | `"BIG"` / `"1234"` — EDID identity used for direct-mode whitelisting |
53	| `dsc_version/slices/bppx16` | 17 (0x11), 4, 128 — Display Stream Compression config |
54	| `display_color_mult` | per-eye RGB white-point multipliers `[0.952545, 0.917645, 1]` (identical both eyes and across all sampled units → design-level) |
55	| `seconds_from_vsync_to_photons` | 0.002 |
56	| `seconds_from_photons_to_vblank` | 0.0138888888888889 |
57	| `parallel_render_cameras` | true |
58	| `head` | head-origin frame (`plus_x`,`plus_z`,`position`) |
59	| `imu` | mount orientation `plus_x [-1,0,0]`, `plus_z [0.00067,0.1961,-0.98058]`, `position`, biases/scale (per-unit if IMU cal ran) |
60	| `ipd` | production B1: `{"default_mm": <unit IPD>}`; EVT/DVT/lhcalib-era: `{"enable": true, "high_mm": 65.0, "low_mm": 65.0, "minstep_mm": 2}` |
61	| `lighthouse_config` | per-unit: `channelMap` (22 entries), `modelPoints`, `modelNormals` |
62	| `tracking_to_eye_transform` | array[2] = LEFT, RIGHT — the optics block, below |
63	| `firmware_config` | `{"imu_rate": 1000}` |
64	| `expected_bases/controllers` | 2/2 |
65	| `bscv` | added by calibot: photodiode-cal version stamp |
66
67	### 3.1 `tracking_to_eye_transform[eye]` — the distortion model
68
69	Per eye:
70
71	- `intrinsics`: 3×3 ideal pinhole projection in Valve convention
72	  `[[fx,0,cx],[0,fy,cy],[0,0,-1]]` with `cx = -center_x`, `cy = ±center_y`. fx,fy ≈ 1.422 (production B1) → per-eye half-FOV = atan(1/1.4224) ≈ 35.1°, i.e. ≈70.3°×70.3° per eye before canting. This is the projection used for the render camera; distortion is defined relative to it.
73	- `distortion`, `distortion_red`, `distortion_blue`: per-color radial polynomial. Fields: `type`, `center_x`, `center_y` (NDC units, shared across the three channels within an eye), `coeffs`.
74	- `eye_to_head`: 3×3 **rotation only** (display canting). Translation comes from `ipd.default_mm` (split symmetrically by the driver — inferred; the JSON nowhere stores an eye translation).
75	- `grow_for_undistort`: 0.6 (production B1; 0.4 EVT/DVT/B2-test) — how much the pre-distortion render target is "grown" so undistorted content covers the panel.
76	- `undistort_r2_cutoff`: 1.0 — clamp on r² beyond which the polynomial is not extrapolated.
77
78	Distortion `type` evolution observed:
79
80	| era | type | coeffs |
81	|---|---|---|
82	| EVT + DVT (`init_json_evt.json`, `DVTs/LHR-*.json`) | `DISTORT_DPOLY3` | 3 per channel, e.g. green L `[-0.2204, -0.0196, 0.0017]` (negative leading) |
83	| Production B1 (`calibot/init_json_config.json`) | `DISTORT_POLY3` | 3 per channel, e.g. green L `[0.2390, 0.0549, 0.1362]` (positive leading), red `[0.2159, 0.0887, 0.1174]`, blue `[0.2805, -0.0059, 0.1701]` |
84	| Beyond 2 test (`big-notebooks/testConfigFlasher.ipynb`) | `DISTORT_DPOLY3_SCALED` | 4: shared base `[k1,k2,k3]` for all channels + 4th coeff = per-channel radial scale: green `0.0`, blue `+0.01`, red `-0.006904` |
85
86	Interpretation (the evaluator itself is inside Valve's closed lighthouse driver; none of these repos contain it):
87
88	- The DPOLY3 family is the long-known Vive-format model. Public reimplementations (Monado `u_compute_distortion_vive`, OpenHMD) evaluate, **mapping panel/output UV → pre-distortion texture UV**, per channel c:
89	  - `t = 2·uv − 1` (per-eye NDC), optionally aspect-corrected
90	  - `t -= (center_x, center_y)`
91	  - `r² = dot(t,t)`, clamped to `undistort_r2_cutoff`
92	  - `d = 1 + k1·r² + k2·r⁴ + k3·r⁶`
93	  - `uv_src = 0.5 + (t·d + center) · 0.5/(1 + grow_for_undistort)` (the grow factor rescales so the grown render target maps onto the panel)
94	  [external knowledge, marked: matches field names exactly — `grow_for_undistort`, `undistort_r2_cutoff`, `center_x/y`, 3 coeffs — and is what Monado uses to drive real Vive/Index headsets from this same JSON]
95	- SPECULATION (consistent magnitudes/signs): `DISTORT_POLY3` (positive coeffs) is the same polynomial fitted in the **opposite direction** (undistort vs distort); `1/(1+0.22 r²) ≈ 1−0.22r²+…` matches the observed sign flip between DVT (−0.22) and production (+0.239). Verify empirically against SteamVR output (see §8 validation).
96	- `DISTORT_DPOLY3_SCALED`: 4th coefficient is a per-channel scale capturing lateral chromatic aberration as pure radial magnification (blue larger, red smaller) on top of a shared base curve. SPECULATION on exact placement: either `d = (1+k4)·(1 + k1r²+k2r⁴+k3r⁶)` or `d = 1+k4 + k1r²+…`; for k4 ≤ 0.01 the two differ negligibly.
97	- Chromatic aberration in POLY3/DPOLY3 variants is encoded as fully separate per-channel polynomials around a shared center.
98
99	### 3.2 Canting (`eye_to_head`) and per-unit alignment
100
101	- `check-canting/check_canting.ipynb`: utility that converts pitch/yaw/roll (degrees; +pitch up, +yaw left, +roll CW, user-relative) ↔ `eye_to_head` matrices via scipy `Rotation.from_euler('xyz')`. Used to author/inspect config values.
102	- Production B1 values: yaw = ±6.17° (outward per eye; `asin(0.10748)`), pitch ≈ +4.5…5.1° in template (per-unit), roll = 0.
103	- EVT/DVT values: yaw = ±2.0° (`asin(0.034855)`), pitch ≈ +5.0°.
104	- VAP1 (Beyond 1 launch era, `vertical-alignment-proximity/backup_vap1_alignment.py::align_determine_horizontal_canting`): horizontal cant **depended on IPD SKU** — 2° if IPD ≥ 70 mm else 4.5°. Current `alignment.py`: constants `DOWNWARD_CANT = -5`, `HORIZONTAL_CANT = 6.17`.
105
106	**VAP station per-unit measurement** (`vertical-alignment-proximity/README.md`, `alignment.py`, `main.py`):
107
108	1. Operator scans HMD serial; it is written into the MCU TLV config (tag 0x08 `HMD_Serial`, `hmd_config.py::hmd_config_write_serial`; the 512-byte TLV store also holds PCB serial 0x01 `XCNL…`, prox cal/threshold/hysteresis 0x06/0x0B/0x0C, brightness 0x0A, etc., read/written via HID feature reports `'U'`/`'W'`/`'V'` with CRC-8 poly 0x07).
109	2. Proximity sensor calibrated (two-position measure, PASS/FAIL, values into MCU TLV).
110	3. Optical center of each lens found by displaying a ChArUco board **directly on the panel** (no SteamVR; `DirectModeDisplay`) and detecting corner ID 169/181 with a fixed 640×480 camera under the HMD.
111	4. A SteamVR-rendered app (DirectX12Seed.exe via named pipe `\\.\pipe\vap_pipe`) draws a horizontal line while sweeping virtual-camera pitch ±7° in 0.1° steps; CV finds the step-edge crossing the optical center; the zero-crossing angle = per-eye pitch error. (Closed-form CV solutions were tried and abandoned as unreliable — README.)
112	5. Writes per-unit result to SIP flash: `eye_to_head[L] = R_xyz(measured_pitch_L + (−5°), +6.17°, 0)`, `eye_to_head[R] = R_xyz(measured_pitch_R + (−5°), −6.17°, 0)` (`alignment.py::align_write_config_values`), plus `ipd.default_mm` (`align_init` → `steamvr.steamvr.set_ipd_default_mm`). VAP1 additionally solved per-eye roll (`backup_vap1_alignment.py`).
113	6. Verification step re-measures transition points and PASS/FAILs (≤10 px disparity). `confirm_distortion()` only checks `distortion_red` blocks exist — distortion itself is never measured per unit.
114
115	So the complete **per-unit** optical data = `eye_to_head` rotations (pitch±, fixed yaw cant, roll 0) + `ipd.default_mm` + serial. Everything else in the optics block is per-design.
116
117	---
118
119	## 4. valve-tools-wrapper — Valve tools inventory
120
121	`valve-tools-wrapper/steamvr/tools/bin/win64/` ships four Valve factory executables:
122
123	- **`lighthouse_console.exe`** — interactive console wrapped by `steamvr/lighthouse_console.py` via stdin/stdout. Commands used: `serial` (list/select device), `deviceinfo` (`DEVICEINFO serial=… device_class=hmd`), `downloadconfig <path>` / `uploadconfig <path>` (SIP flash JSON), `pair`, `reboot`, `poweroff`, `haptic2`, `userdata`. This is the read/write path for **all** calibration data on the device.
124	- **`lhcalib.exe`** — photodiode + IMU calibration solver (args in §2; also `/imucal <seconds> /imucalautoexit`, `/usedisambiguation synconbeam`, `/outputdir`). Emits `auto_<serial>.json` (solved config) and `calib_observations_<base>_<serial>.json` (raw observations: `file_tracking`, `per_base`, `primary_basestation`, `tracked_device_config`).
125	- **`imu_calibrator.exe`**, **`vrtrackingcalib.exe`** — present but not referenced by the Python code in these repos.
126
127	`steamvr/steamvr.py` extras:
128
129	- `update_steam_real_time(config)` → `vrcmd.exe --cmdhmd "set_lenscal <json>"` — **live-reloads lens calibration into a running SteamVR lighthouse driver**. Direct proof the driver consumes exactly this JSON for distortion.
130	- `load/save_steamvr_vrsettings()` and `SteamVR.set_ipd()` → writes `steamvr.ipdOffset` (meters, relative to 63 mm default) into `C:\Program Files (x86)\Steam\config\steamvr.vrsettings`.
131	- Config mutators: `set_resolution`, `set_serial_number`, `set_ipd_default_mm`, `swap_eyes`, `swap_xy_distortion_center`, `set_imu_calibration`.
132	- `steamvr/unbuffered/` — non-blocking variants + `DeviceMonitor` (HID hotplug watcher used by calibot).
133
134	---
135
136	## 5. big-notebooks — what's relevant
137
138	Mostly Shopify/fulfillment ops against the Bigscreen admin cloud (`api.py`: auth/admin/cloud API, `/admin/fabricator/*`, `/admin/shop/*`). Calibration-relevant items:
139
140	- **`testConfigFlasher.ipynb`** — flashes `BS2_TEST_CONFIG.json` (Beyond 2 test mule) to a SIP via lighthouse_console; its full output (cell 4) is the complete B2 test config (source of the `DISTORT_DPOLY3_SCALED` model, §3.1).
141	- **IPD pipeline**: customer face-scan → `scanRequest.ipd` / re-derived "far PD" (`040.RepairTopologyIPDs.ipynb`, `043.…Part2`), clamped to **55–72 mm, 70 mapped to 69** (no 70 mm SKU). HMDs are **built as discrete IPD SKUs**: inventory queried by `serialNumberLike=BS1{ipd}` (`factoryKit.py::loadBs1Stock`) — the IPD is encoded in the HMD serial digits [3:5], which is exactly what VAP1 parsed (`backup_vap1_main.py: input_ipd = int(serial_number[3:5])`). Production quotas per SKU in `022.hmdFulfillment.ipynb`.
142	- `023.hmdTesting.ipynb` — controller pairing/haptics via lighthouse_console; `startSteamVR.py`.
143	- No optics-math notebooks here; the optics math lives in `check-canting` and VAP (§3.2).
144
145	---
146
147	## 6. IPD handling summary (Beyond 1 vs Beyond 2)
148
149	- **Beyond 1**: fixed-IPD hardware SKU (lens spacing built-in, 55–72 mm, no 70). Per-unit `ipd.default_mm` written at VAP from the serial-encoded SKU; `eye_to_head` carries no translation (rotation only) — the driver derives eye translation from `ipd.default_mm`. Software trim via `steamvr.vrsettings: ipdOffset`. Early units: horizontal cant varied by SKU (4.5°/2°), later 6.17° uniform.
150	- **Beyond 2** (test config + current VAP `main.py`): `align_init(serial, 64.0, …)` — IPD hardcoded to 64.0 at the factory. B2 has user-adjustable IPD hardware [public knowledge]; SPECULATION: actual IPD is applied at runtime (user setting / sensor) rather than baked per unit; the config's 64 is just a default. The B2 distortion model moved to shared-base + per-channel scale (`DISTORT_DPOLY3_SCALED`), suggesting one curve valid across the IPD adjustment range with CA as scale.
151
152	---
153
154	## 7. Where everything lives (per-unit data access paths)
155
156	| data | store | access |
157	|---|---|---|
158	| Full optical+tracking JSON (distortion, intrinsics, eye_to_head, ipd, lighthouse_config, imu, timings, color mult) | Tundra SIP flash (zlib) | `lighthouse_console.exe` `downloadconfig`/`uploadconfig` (USB HID, VID 0x28de PID 0x2300); SteamVR auto-caches to `Steam\config\lighthouse\<lhr-serial>\config.json` (lowercased serial) |
159	| Mura/banding correction | device userdata → SteamVR cache `Steam\config\lighthouse\<lhr-serial>\userdata\mura.mc` | PNG-format multiply mask; `extended-mode-display/bars_gen.py` generates column-banding corrections (linear-space multiply, per channel) and overwrites `mura.mc` directly |
160	| Prox cal/threshold/hysteresis, HMD serial, brightness, PCB serial | Beyond MCU 512-B TLV config | HID feature reports `'U'` read / `'W'` write / `'V'` save (VID 0x35bd PID 0x0101), `vertical-alignment-proximity/hmd_config.py` |
161	| Order ↔ IPD ↔ face-scan | Bigscreen cloud | `big-notebooks/api.py` admin API (not needed for rendering) |
162	| PASS/FAIL test records | factory MES (LAN) | `mes_logger.py` |
163
164	SteamVR consumption: lighthouse driver reads SIP JSON → implements `ComputeDistortion()` (per-channel poly3), projection from `intrinsics`, eye poses from `eye_to_head`+`ipd`, display timing from the `seconds_*` fields; compositor builds warp mesh by sampling the driver, applies `display_color_mult` and `mura.mc`. Live reload hook: `vrcmd --cmdhmd "set_lenscal <json>"`.
165
166	---
167
168	## 8. Reimplementing distortion correction WITHOUT SteamVR
169
170	Required inputs (all obtainable from the unit itself, offline):
171
172	1. The device JSON — either dump with `valve-tools-wrapper/steamvr/tools/bin/win64/lighthouse_console.exe` (`downloadconfig`), or just read SteamVR's cache `Steam\config\lighthouse\<serial>\config.json`. One-time per unit; no cloud, no lighthouses needed.
173	2. `mura.mc` from the same cache dir (optional quality step).
174	3. The polynomial evaluator (§3.1) — implement Monado-equivalent `compute_distortion(eye, u, v) → (uv_r, uv_g, uv_b)`.
175
176	Render pipeline per eye:
177
178	1. Render scene with projection from `intrinsics` (half-FOV_x = atan(1/fx), centers from cx,cy; asymmetric frustum), camera pose = head_pose · eye_to_head_rotation, eye offset = ±ipd.default_mm/2 (sign per eye) along head +X.
179	2. Oversample render target by `(1 + grow_for_undistort)` relative to 2544×2544 panel area.
180	3. Full-screen warp pass (mesh or fragment): for each panel pixel, evaluate the per-channel polynomial to get three source UVs; sample R/G/B separately (this is both distortion correction and lateral CA correction).
181	4. Multiply by `display_color_mult` per eye and the `mura.mc` mask (in linear light — `bars_gen.py` applies corrections in linear space).
182
183	**Validation strategy** (resolves the POLY3 direction question empirically): run SteamVR once, use `vrcmd`/OpenVR `GetDistortionMesh`/`ComputeDistortion` to dump Valve's ground-truth UV mapping for this unit, and fit/verify the standalone evaluator against it. Alternatively diff rendered output photographically through the lens. The EVT/DVT-vs-production sign flip (−0.22 vs +0.239) makes blind implementation risky; one dump of ground truth removes all ambiguity.
184
185	---
186
187	## Relevance to sauna
188
189	Concrete recipe for a standalone (no-SteamVR, no-lighthouse) wearable-display renderer for a given Beyond unit:
190
191	1. **Extract per-unit calibration once.** Plug in HMD, run `lighthouse_console.exe downloadconfig beyond.json` (admin shell; tool in `valve-tools-wrapper/steamvr/tools/bin/win64/`). Grab `mura.mc` from `Steam\config\lighthouse\<serial>\userdata\` if present. This file contains everything: per-channel distortion polynomials, intrinsics, per-unit eye_to_head, IPD, panel timings, color mult, IMU mount frame + biases.
192	2. **Get pixels to the panel without SteamVR** — two proven first-party paths in `extended-mode-display`:
193	   - *Extended mode*: MCU HID feature report `[0,'d',rate]` (`rate` 2=75 Hz tested, 1=90 Hz "at own risk") rewrites the linkbox EDID so the HMD appears as a normal 3840×1920 side-by-side monitor (1920×1920/eye — reduced from native 2560×2560); then any fullscreen window works (`ExtendedModeDisplay`, cv2). Restore with the same command. Also needed: display power `'+'`, panel on `[0,'o',eye,1,0x29]`, duty `'I'`, prox disable `'p'`.
194	   - *Direct mode at native res*: `resources/direct_mode_dx12.exe` (`-enable/-disable`, `-dsc_present_test 0 1 1 0 0x11 4 128` matching the config's DSC fields; env `DM_VENDOR_ID=0x2709`) and `display_image.exe` present full-res images. Harder but full 2544/2560 px.
195	3. **Implement the warp** as a single post pass (§8): per-channel poly3 UV remap + grow factor + color mult (+ optional mura multiply). ~50 lines of GLSL/HLSL once coefficients are loaded.
196	4. **3DOF pose** from the onboard ICM-class IMU (1 kHz, `firmware_config.imu_rate`): orientation via any AHRS filter, using `imu.plus_x/plus_z/position` to rotate IMU frame → head frame, and config gyro/acc biases. The IMU streams over the Tundra SIP HID interface (watchman protocol — parse per OpenHMD/libsurvive's watchman IMU report layout; protocol itself is not in these repos).
197	5. **6DOF (stretch)**: the 22 photodiodes are useless without base stations, so 6DOF needs an external method (e.g. camera/inside-out or phone-assisted). The `lighthouse_config.modelPoints` do give you the exact sensor constellation geometry if optical fiducials are ever wanted.
198	6. **Honor per-eye view setup**: render cameras rotated by `eye_to_head` (≈ ±6.17° yaw, −5°+δ pitch per unit) and separated by `ipd.default_mm` — skipping this (not just distortion) is what breaks binocular fusion on canted-display HMDs.
199	7. **Calibration sanity check**: before cutting SteamVR loose, dump ground-truth warp UVs from OpenVR (`ComputeDistortion` per eye on a grid) on the same unit and unit-test the standalone evaluator against them (settles POLY3-vs-DPOLY3 direction and the `grow_for_undistort` normalization exactly).
200

# Bigscreen Beyond — Calibration & Distortion Pipeline

Sources (all first-party, read-only; branch `main` unless noted):

- `calibot` — C:\Users\decid\Documents\projects\calibot (factory tracking-calibration rig)
- `valve-tools-wrapper` — C:\Users\decid\Documents\projects\bigscreen\valve-tools-wrapper (Python wrapper around Valve factory tools + sample per-device configs)
- `big-notebooks` — C:\Users\decid\Documents\projects\bigscreen\big-notebooks (ops/fulfillment notebooks + config flasher)
- Supplementary (found adjacent, directly relevant): `vertical-alignment-proximity` (branch `master`) — C:\Users\decid\Documents\projects\bigscreen\vertical-alignment-proximity (per-unit optical alignment station, "VAP"); `check-canting` — C:\Users\decid\Documents\projects\check-canting; `extended-mode-display` — C:\Users\decid\Documents\projects\extended-mode-display (non-SteamVR display access).

---

## 1. The big picture

The Beyond is, to SteamVR, a **stock lighthouse HMD**. Its tracking SIP is a Tundra Labs module enumerating with Valve's VID/PID (`0x28de:0x2300`, see `calibot/calibrate_diode.py` `VIDS_TUNDRA/PIDS_TUNDRA`); a separate Bigscreen MCU ("Atmel", `0x35bd:0x0101`, product string "Beyond") handles displays/LED/proximity/fan. **All optical + tracking calibration lives in a single Valve-format JSON config stored in the tracking SIP's flash** (~2 KB zlib-compressed, ~11 KB inflated — see notebook output in `big-notebooks/testConfigFlasher.ipynb`: "Read config of 1979 bytes ... inflated to 11004 bytes"). SteamVR's built-in lighthouse driver downloads that JSON from the device at startup, caches it under `C:\Program Files (x86)\Steam\config\lighthouse\<lhr-serial>\`, and implements `IVRDisplayComponent::ComputeDistortion()` from its `tracking_to_eye_transform` block. The compositor samples that function to build its warp mesh. No Bigscreen-authored OpenVR driver exists in any of these repos — the distortion consumer is Valve's closed `driver_lighthouse`.

Factory chain per unit:

1. **calibot station** — flashes a full template config (incl. design-level distortion polynomials) to the SIP, then solves per-unit photodiode geometry with Valve's `lhcalib.exe` and writes it back to SIP flash.
2. **VAP station** — calibrates the proximity sensor (into the MCU's 512-byte TLV config memory) and measures per-unit display/optics rotational misalignment with a camera; writes per-unit `eye_to_head` matrices and `ipd.default_mm` into the SIP JSON config.
3. Results PASS/FAIL go to a factory MES web endpoint (`vertical-alignment-proximity/mes_logger.py`, station IDs "VAP T060, MMI T070, BT T080"). The Bigscreen cloud (`big-notebooks/api.py`) stores order/IPD/face-scan data, **not** optical calibration; no evidence of per-serial calibration in the cloud.

---

## 2. calibot — what it is

`calibot/README.md`: "Software run on calibot - used for steamVR tracking". It is a **2-axis motorized motion rig** (two Trinamic TMCM-1161 stepper modules over serial-TMCL, worm ratio 180:1, axes addressed by global-parameter IDs 159/160 — `calibot/calibot.py`). It slings up to **4 headsets at once** through a randomized motion path in view of **2–3 lighthouse base stations** (`calibot/calibot_config.yaml`: `basestation_count: 3`).

Flow (`calibot/calibrate_diode.py`, the shipping app; `calibot/main.py` is an older variant):

1. USB device-monitor matches each Atmel MCU to its Tundra SIP by shared USB hub port chain (`HeadsetHIDMatching`); headset LED is driven via MCU HID feature report `[0,'L',r,g,b]` for operator status (white=in progress, green=pass, red=fail).
2. `calibot/load_init_json.py` uploads `calibot/init_json_config.json` **wholesale** to the SIP via `lighthouse_console.exe uploadconfig`, with only `device_serial_number` substituted per device. I.e. at this stage **every unit receives identical distortion polynomials** — the optical distortion model is per-design (possibly per lens revision), not measured per unit.
3. Runs Valve's `lhcalib.exe`:
   `lhcalib /bodyserial <LHR-…> /bodycal init_json_config.json <total_hits=800> <hits_per_sensor=80> /bodycalmulti <n_bases> /deletemissingsensors`
   which solves photodiode positions/normals from sweep observations during arm motion. Output file `auto_<serial>.json` echoes the input config with solved `lighthouse_config` (`channelMap` [22 sensors], `modelPoints` [m, HMD frame], `modelNormals`).
4. The station downloads the device's current config, replaces only `lighthouse_config`, adds a `bscv` version key (`"<total_hits>.<hits_per_sensor>"`, e.g. `800.80`), and re-uploads (`calibrate_headset()` in `calibot/calibrate_diode.py`).
5. IMU calibration support exists (`ImuCalibrationProcess` → `acc_bias`, `acc_scale`, `gyro_bias`) but is commented out in `calibot/main.py::create_imu_calibration_threads` — production template ships zero biases.

So calibot measures: **photodiode geometry per unit** (tracking), optionally IMU bias. It does **not** measure optics.

---

## 3. The on-device config JSON (the calibration artifact)

Canonical template: `calibot/init_json_config.json` (production Beyond 1, `revision: 0.12`). Older: `calibot/init_json_config_o.json` (differs only in serial placeholder, `ipd.default_mm` 63 vs 73, and slightly different nominal `modelPoints`); `valve-tools-wrapper/steamvr/init_json_evt.json` (EVT era); 20 real DVT-unit dumps in `valve-tools-wrapper/steamvr/tools/bin/win64/DVTs/LHR-*.json`; a Beyond-2 test config embedded in `big-notebooks/testConfigFlasher.ipynb` (`BS2_TEST_CONFIG.json`, file itself not in repo).

Top-level keys (all variants):

| key | meaning / production value |
|---|---|
| `manufacturer`/`model_number` | "Bigscreen"/"Beyond" |
| `device` | `eye_target_width/height_in_pixels: 2544×2544` per eye, `first_eye eEYE_LEFT`, `num_windows: 1` |
| `direct_mode_edid_vid/pid` | `"BIG"` / `"1234"` — EDID identity used for direct-mode whitelisting |
| `dsc_version/slices/bppx16` | 17 (0x11), 4, 128 — Display Stream Compression config |
| `display_color_mult` | per-eye RGB white-point multipliers `[0.952545, 0.917645, 1]` (identical both eyes and across all sampled units → design-level) |
| `seconds_from_vsync_to_photons` | 0.002 |
| `seconds_from_photons_to_vblank` | 0.0138888888888889 |
| `parallel_render_cameras` | true |
| `head` | head-origin frame (`plus_x`,`plus_z`,`position`) |
| `imu` | mount orientation `plus_x [-1,0,0]`, `plus_z [0.00067,0.1961,-0.98058]`, `position`, biases/scale (per-unit if IMU cal ran) |
| `ipd` | production B1: `{"default_mm": <unit IPD>}`; EVT/DVT/lhcalib-era: `{"enable": true, "high_mm": 65.0, "low_mm": 65.0, "minstep_mm": 2}` |
| `lighthouse_config` | per-unit: `channelMap` (22 entries), `modelPoints`, `modelNormals` |
| `tracking_to_eye_transform` | array[2] = LEFT, RIGHT — the optics block, below |
| `firmware_config` | `{"imu_rate": 1000}` |
| `expected_bases/controllers` | 2/2 |
| `bscv` | added by calibot: photodiode-cal version stamp |

### 3.1 `tracking_to_eye_transform[eye]` — the distortion model

Per eye:

- `intrinsics`: 3×3 ideal pinhole projection in Valve convention
  `[[fx,0,cx],[0,fy,cy],[0,0,-1]]` with `cx = -center_x`, `cy = ±center_y`. fx,fy ≈ 1.422 (production B1) → per-eye half-FOV = atan(1/1.4224) ≈ 35.1°, i.e. ≈70.3°×70.3° per eye before canting. This is the projection used for the render camera; distortion is defined relative to it.
- `distortion`, `distortion_red`, `distortion_blue`: per-color radial polynomial. Fields: `type`, `center_x`, `center_y` (NDC units, shared across the three channels within an eye), `coeffs`.
- `eye_to_head`: 3×3 **rotation only** (display canting). Translation comes from `ipd.default_mm` (split symmetrically by the driver — inferred; the JSON nowhere stores an eye translation).
- `grow_for_undistort`: 0.6 (production B1; 0.4 EVT/DVT/B2-test) — how much the pre-distortion render target is "grown" so undistorted content covers the panel.
- `undistort_r2_cutoff`: 1.0 — clamp on r² beyond which the polynomial is not extrapolated.

Distortion `type` evolution observed:

| era | type | coeffs |
|---|---|---|
| EVT + DVT (`init_json_evt.json`, `DVTs/LHR-*.json`) | `DISTORT_DPOLY3` | 3 per channel, e.g. green L `[-0.2204, -0.0196, 0.0017]` (negative leading) |
| Production B1 (`calibot/init_json_config.json`) | `DISTORT_POLY3` | 3 per channel, e.g. green L `[0.2390, 0.0549, 0.1362]` (positive leading), red `[0.2159, 0.0887, 0.1174]`, blue `[0.2805, -0.0059, 0.1701]` |
| Beyond 2 test (`big-notebooks/testConfigFlasher.ipynb`) | `DISTORT_DPOLY3_SCALED` | 4: shared base `[k1,k2,k3]` for all channels + 4th coeff = per-channel radial scale: green `0.0`, blue `+0.01`, red `-0.006904` |

Interpretation (the evaluator itself is inside Valve's closed lighthouse driver; none of these repos contain it):

- The DPOLY3 family is the long-known Vive-format model. Public reimplementations (Monado `u_compute_distortion_vive`, OpenHMD) evaluate, **mapping panel/output UV → pre-distortion texture UV**, per channel c:
  - `t = 2·uv − 1` (per-eye NDC), optionally aspect-corrected
  - `t -= (center_x, center_y)`
  - `r² = dot(t,t)`, clamped to `undistort_r2_cutoff`
  - `d = 1 + k1·r² + k2·r⁴ + k3·r⁶`
  - `uv_src = 0.5 + (t·d + center) · 0.5/(1 + grow_for_undistort)` (the grow factor rescales so the grown render target maps onto the panel)
  [external knowledge, marked: matches field names exactly — `grow_for_undistort`, `undistort_r2_cutoff`, `center_x/y`, 3 coeffs — and is what Monado uses to drive real Vive/Index headsets from this same JSON]
- SPECULATION (consistent magnitudes/signs): `DISTORT_POLY3` (positive coeffs) is the same polynomial fitted in the **opposite direction** (undistort vs distort); `1/(1+0.22 r²) ≈ 1−0.22r²+…` matches the observed sign flip between DVT (−0.22) and production (+0.239). Verify empirically against SteamVR output (see §8 validation).
- `DISTORT_DPOLY3_SCALED`: 4th coefficient is a per-channel scale capturing lateral chromatic aberration as pure radial magnification (blue larger, red smaller) on top of a shared base curve. SPECULATION on exact placement: either `d = (1+k4)·(1 + k1r²+k2r⁴+k3r⁶)` or `d = 1+k4 + k1r²+…`; for k4 ≤ 0.01 the two differ negligibly.
- Chromatic aberration in POLY3/DPOLY3 variants is encoded as fully separate per-channel polynomials around a shared center.

### 3.2 Canting (`eye_to_head`) and per-unit alignment

- `check-canting/check_canting.ipynb`: utility that converts pitch/yaw/roll (degrees; +pitch up, +yaw left, +roll CW, user-relative) ↔ `eye_to_head` matrices via scipy `Rotation.from_euler('xyz')`. Used to author/inspect config values.
- Production B1 values: yaw = ±6.17° (outward per eye; `asin(0.10748)`), pitch ≈ +4.5…5.1° in template (per-unit), roll = 0.
- EVT/DVT values: yaw = ±2.0° (`asin(0.034855)`), pitch ≈ +5.0°.
- VAP1 (Beyond 1 launch era, `vertical-alignment-proximity/backup_vap1_alignment.py::align_determine_horizontal_canting`): horizontal cant **depended on IPD SKU** — 2° if IPD ≥ 70 mm else 4.5°. Current `alignment.py`: constants `DOWNWARD_CANT = -5`, `HORIZONTAL_CANT = 6.17`.

**VAP station per-unit measurement** (`vertical-alignment-proximity/README.md`, `alignment.py`, `main.py`):

1. Operator scans HMD serial; it is written into the MCU TLV config (tag 0x08 `HMD_Serial`, `hmd_config.py::hmd_config_write_serial`; the 512-byte TLV store also holds PCB serial 0x01 `XCNL…`, prox cal/threshold/hysteresis 0x06/0x0B/0x0C, brightness 0x0A, etc., read/written via HID feature reports `'U'`/`'W'`/`'V'` with CRC-8 poly 0x07).
2. Proximity sensor calibrated (two-position measure, PASS/FAIL, values into MCU TLV).
3. Optical center of each lens found by displaying a ChArUco board **directly on the panel** (no SteamVR; `DirectModeDisplay`) and detecting corner ID 169/181 with a fixed 640×480 camera under the HMD.
4. A SteamVR-rendered app (DirectX12Seed.exe via named pipe `\\.\pipe\vap_pipe`) draws a horizontal line while sweeping virtual-camera pitch ±7° in 0.1° steps; CV finds the step-edge crossing the optical center; the zero-crossing angle = per-eye pitch error. (Closed-form CV solutions were tried and abandoned as unreliable — README.)
5. Writes per-unit result to SIP flash: `eye_to_head[L] = R_xyz(measured_pitch_L + (−5°), +6.17°, 0)`, `eye_to_head[R] = R_xyz(measured_pitch_R + (−5°), −6.17°, 0)` (`alignment.py::align_write_config_values`), plus `ipd.default_mm` (`align_init` → `steamvr.steamvr.set_ipd_default_mm`). VAP1 additionally solved per-eye roll (`backup_vap1_alignment.py`).
6. Verification step re-measures transition points and PASS/FAILs (≤10 px disparity). `confirm_distortion()` only checks `distortion_red` blocks exist — distortion itself is never measured per unit.

So the complete **per-unit** optical data = `eye_to_head` rotations (pitch±, fixed yaw cant, roll 0) + `ipd.default_mm` + serial. Everything else in the optics block is per-design.

---

## 4. valve-tools-wrapper — Valve tools inventory

`valve-tools-wrapper/steamvr/tools/bin/win64/` ships four Valve factory executables:

- **`lighthouse_console.exe`** — interactive console wrapped by `steamvr/lighthouse_console.py` via stdin/stdout. Commands used: `serial` (list/select device), `deviceinfo` (`DEVICEINFO serial=… device_class=hmd`), `downloadconfig <path>` / `uploadconfig <path>` (SIP flash JSON), `pair`, `reboot`, `poweroff`, `haptic2`, `userdata`. This is the read/write path for **all** calibration data on the device.
- **`lhcalib.exe`** — photodiode + IMU calibration solver (args in §2; also `/imucal <seconds> /imucalautoexit`, `/usedisambiguation synconbeam`, `/outputdir`). Emits `auto_<serial>.json` (solved config) and `calib_observations_<base>_<serial>.json` (raw observations: `file_tracking`, `per_base`, `primary_basestation`, `tracked_device_config`).
- **`imu_calibrator.exe`**, **`vrtrackingcalib.exe`** — present but not referenced by the Python code in these repos.

`steamvr/steamvr.py` extras:

- `update_steam_real_time(config)` → `vrcmd.exe --cmdhmd "set_lenscal <json>"` — **live-reloads lens calibration into a running SteamVR lighthouse driver**. Direct proof the driver consumes exactly this JSON for distortion.
- `load/save_steamvr_vrsettings()` and `SteamVR.set_ipd()` → writes `steamvr.ipdOffset` (meters, relative to 63 mm default) into `C:\Program Files (x86)\Steam\config\steamvr.vrsettings`.
- Config mutators: `set_resolution`, `set_serial_number`, `set_ipd_default_mm`, `swap_eyes`, `swap_xy_distortion_center`, `set_imu_calibration`.
- `steamvr/unbuffered/` — non-blocking variants + `DeviceMonitor` (HID hotplug watcher used by calibot).

---

## 5. big-notebooks — what's relevant

Mostly Shopify/fulfillment ops against the Bigscreen admin cloud (`api.py`: auth/admin/cloud API, `/admin/fabricator/*`, `/admin/shop/*`). Calibration-relevant items:

- **`testConfigFlasher.ipynb`** — flashes `BS2_TEST_CONFIG.json` (Beyond 2 test mule) to a SIP via lighthouse_console; its full output (cell 4) is the complete B2 test config (source of the `DISTORT_DPOLY3_SCALED` model, §3.1).
- **IPD pipeline**: customer face-scan → `scanRequest.ipd` / re-derived "far PD" (`040.RepairTopologyIPDs.ipynb`, `043.…Part2`), clamped to **55–72 mm, 70 mapped to 69** (no 70 mm SKU). HMDs are **built as discrete IPD SKUs**: inventory queried by `serialNumberLike=BS1{ipd}` (`factoryKit.py::loadBs1Stock`) — the IPD is encoded in the HMD serial digits [3:5], which is exactly what VAP1 parsed (`backup_vap1_main.py: input_ipd = int(serial_number[3:5])`). Production quotas per SKU in `022.hmdFulfillment.ipynb`.
- `023.hmdTesting.ipynb` — controller pairing/haptics via lighthouse_console; `startSteamVR.py`.
- No optics-math notebooks here; the optics math lives in `check-canting` and VAP (§3.2).

---

## 6. IPD handling summary (Beyond 1 vs Beyond 2)

- **Beyond 1**: fixed-IPD hardware SKU (lens spacing built-in, 55–72 mm, no 70). Per-unit `ipd.default_mm` written at VAP from the serial-encoded SKU; `eye_to_head` carries no translation (rotation only) — the driver derives eye translation from `ipd.default_mm`. Software trim via `steamvr.vrsettings: ipdOffset`. Early units: horizontal cant varied by SKU (4.5°/2°), later 6.17° uniform.
- **Beyond 2** (test config + current VAP `main.py`): `align_init(serial, 64.0, …)` — IPD hardcoded to 64.0 at the factory. B2 has user-adjustable IPD hardware [public knowledge]; SPECULATION: actual IPD is applied at runtime (user setting / sensor) rather than baked per unit; the config's 64 is just a default. The B2 distortion model moved to shared-base + per-channel scale (`DISTORT_DPOLY3_SCALED`), suggesting one curve valid across the IPD adjustment range with CA as scale.

---

## 7. Where everything lives (per-unit data access paths)

| data | store | access |
|---|---|---|
| Full optical+tracking JSON (distortion, intrinsics, eye_to_head, ipd, lighthouse_config, imu, timings, color mult) | Tundra SIP flash (zlib) | `lighthouse_console.exe` `downloadconfig`/`uploadconfig` (USB HID, VID 0x28de PID 0x2300); SteamVR auto-caches to `Steam\config\lighthouse\<lhr-serial>\config.json` (lowercased serial) |
| Mura/banding correction | device userdata → SteamVR cache `Steam\config\lighthouse\<lhr-serial>\userdata\mura.mc` | PNG-format multiply mask; `extended-mode-display/bars_gen.py` generates column-banding corrections (linear-space multiply, per channel) and overwrites `mura.mc` directly |
| Prox cal/threshold/hysteresis, HMD serial, brightness, PCB serial | Beyond MCU 512-B TLV config | HID feature reports `'U'` read / `'W'` write / `'V'` save (VID 0x35bd PID 0x0101), `vertical-alignment-proximity/hmd_config.py` |
| Order ↔ IPD ↔ face-scan | Bigscreen cloud | `big-notebooks/api.py` admin API (not needed for rendering) |
| PASS/FAIL test records | factory MES (LAN) | `mes_logger.py` |

SteamVR consumption: lighthouse driver reads SIP JSON → implements `ComputeDistortion()` (per-channel poly3), projection from `intrinsics`, eye poses from `eye_to_head`+`ipd`, display timing from the `seconds_*` fields; compositor builds warp mesh by sampling the driver, applies `display_color_mult` and `mura.mc`. Live reload hook: `vrcmd --cmdhmd "set_lenscal <json>"`.

---

## 8. Reimplementing distortion correction WITHOUT SteamVR

Required inputs (all obtainable from the unit itself, offline):

1. The device JSON — either dump with `valve-tools-wrapper/steamvr/tools/bin/win64/lighthouse_console.exe` (`downloadconfig`), or just read SteamVR's cache `Steam\config\lighthouse\<serial>\config.json`. One-time per unit; no cloud, no lighthouses needed.
2. `mura.mc` from the same cache dir (optional quality step).
3. The polynomial evaluator (§3.1) — implement Monado-equivalent `compute_distortion(eye, u, v) → (uv_r, uv_g, uv_b)`.

Render pipeline per eye:

1. Render scene with projection from `intrinsics` (half-FOV_x = atan(1/fx), centers from cx,cy; asymmetric frustum), camera pose = head_pose · eye_to_head_rotation, eye offset = ±ipd.default_mm/2 (sign per eye) along head +X.
2. Oversample render target by `(1 + grow_for_undistort)` relative to 2544×2544 panel area.
3. Full-screen warp pass (mesh or fragment): for each panel pixel, evaluate the per-channel polynomial to get three source UVs; sample R/G/B separately (this is both distortion correction and lateral CA correction).
4. Multiply by `display_color_mult` per eye and the `mura.mc` mask (in linear light — `bars_gen.py` applies corrections in linear space).

**Validation strategy** (resolves the POLY3 direction question empirically): run SteamVR once, use `vrcmd`/OpenVR `GetDistortionMesh`/`ComputeDistortion` to dump Valve's ground-truth UV mapping for this unit, and fit/verify the standalone evaluator against it. Alternatively diff rendered output photographically through the lens. The EVT/DVT-vs-production sign flip (−0.22 vs +0.239) makes blind implementation risky; one dump of ground truth removes all ambiguity.

---

## Relevance to sauna

Concrete recipe for a standalone (no-SteamVR, no-lighthouse) wearable-display renderer for a given Beyond unit:

1. **Extract per-unit calibration once.** Plug in HMD, run `lighthouse_console.exe downloadconfig beyond.json` (admin shell; tool in `valve-tools-wrapper/steamvr/tools/bin/win64/`). Grab `mura.mc` from `Steam\config\lighthouse\<serial>\userdata\` if present. This file contains everything: per-channel distortion polynomials, intrinsics, per-unit eye_to_head, IPD, panel timings, color mult, IMU mount frame + biases.
2. **Get pixels to the panel without SteamVR** — two proven first-party paths in `extended-mode-display`:
   - *Extended mode*: MCU HID feature report `[0,'d',rate]` (`rate` 2=75 Hz tested, 1=90 Hz "at own risk") rewrites the linkbox EDID so the HMD appears as a normal 3840×1920 side-by-side monitor (1920×1920/eye — reduced from native 2560×2560); then any fullscreen window works (`ExtendedModeDisplay`, cv2). Restore with the same command. Also needed: display power `'+'`, panel on `[0,'o',eye,1,0x29]`, duty `'I'`, prox disable `'p'`.
   - *Direct mode at native res*: `resources/direct_mode_dx12.exe` (`-enable/-disable`, `-dsc_present_test 0 1 1 0 0x11 4 128` matching the config's DSC fields; env `DM_VENDOR_ID=0x2709`) and `display_image.exe` present full-res images. Harder but full 2544/2560 px.
3. **Implement the warp** as a single post pass (§8): per-channel poly3 UV remap + grow factor + color mult (+ optional mura multiply). ~50 lines of GLSL/HLSL once coefficients are loaded.
4. **3DOF pose** from the onboard ICM-class IMU (1 kHz, `firmware_config.imu_rate`): orientation via any AHRS filter, using `imu.plus_x/plus_z/position` to rotate IMU frame → head frame, and config gyro/acc biases. The IMU streams over the Tundra SIP HID interface (watchman protocol — parse per OpenHMD/libsurvive's watchman IMU report layout; protocol itself is not in these repos).
5. **6DOF (stretch)**: the 22 photodiodes are useless without base stations, so 6DOF needs an external method (e.g. camera/inside-out or phone-assisted). The `lighthouse_config.modelPoints` do give you the exact sensor constellation geometry if optical fiducials are ever wanted.
6. **Honor per-eye view setup**: render cameras rotated by `eye_to_head` (≈ ±6.17° yaw, −5°+δ pitch per unit) and separated by `ipd.default_mm` — skipping this (not just distortion) is what breaks binocular fusion on canted-display HMDs.
7. **Calibration sanity check**: before cutting SteamVR loose, dump ground-truth warp UVs from OpenVR (`ComputeDistortion` per eye on a grid) on the same unit and unit-test the standalone evaluator against them (settles POLY3-vs-DPOLY3 direction and the `grow_for_undistort` normalization exactly).


PowerShell

Get-ChildItem C:\Users\decid\Documents\projects\sauna | Select-Object Name; Get-ChildItem C:\Users\decid\Documents\projects\sauna\reference | Select-Object Name

List repo root and reference dir


Name
----
.claude
docs
reference
spikes
.gitignore
CONTEXT.md
PLAN.md
BASELINE.md
beyond-synaptics.md
calibration-distortion.md
extended-mode-display.md
eye-tracking.md
firmware-update-chain.md
hmdutility.md
openvr-host-tools.md
openvr-resilient.md


No matches found


Test-Path "C:\Program Files (x86)\Steam\config\lighthouse"; if (Test-Path "C:\Program Files (x86)\Steam\config\lighthouse") { Get-ChildItem "C:\Program Files (x86)\Steam\config\lighthouse" -Directory | Select-Object Name }; Test-Path "C:\Program Files (x86)\Steam\steamapps\common\SteamVR"

Check SteamVR install and lighthouse config cache


True

Name
----
lhr-01fd766b
lhr-068e6a77
lhr-13231c27
lhr-1796b086
lhr-17e856e5
lhr-184b2a84
lhr-1bc3ad9c
lhr-1d6f8e15
lhr-1f8e25f1
lhr-211d96a8
lhr-24f77238
lhr-28ce48bd
lhr-2be6de1b
lhr-2d2ba61d
lhr-3ec274e5
lhr-3f0913f3
lhr-4b04c85c
lhr-4c923019
lhr-4c99f398
lhr-51002f4e
lhr-55511d37
lhr-58e9d2d6
lhr-5b70f536
lhr-5c57b08f
lhr-644de747
lhr-6ca7a77d
lhr-86986b50
lhr-881e7037
lhr-882c7f59
lhr-88380134
lhr-9cbc097e
lhr-a8ea7e54
lhr-ab62f94c
lhr-adb4e6c5
lhr-b32aafce
lhr-b5518afd
lhr-b7915abd
lhr-c4dd6b02
lhr-c86ecec9
lhr-d476be21
lhr-e5d235bb
lhr-ebdc0c87
lhr-f556b0a0
lhr-f93128e5
lhr-f998d1aa
lhr-fb07a781
lhr-fdffab6a
True


243:  `lighthouse_console.exe downloadconfig` (≈20 s tracking loss); cached thereafter.
245:- IPD **persistence** to headset flash uses `lighthouse_console.exe download/upload` of the
300:  doesn't — they fall back to `lighthouse_console.exe`). Vendor ref:
306:    directly — it goes through SteamVR's tracking and `lighthouse_console.exe`.
321:  shared mode), `lighthouse_console.exe` (Tundra/Valve config tool, `downloadconfig`/
372:  tracking + `lighthouse_console.exe`. This is the crux for 3DOF/6DOF-without-lighthouses and


SteamVR installed, 47 cached unit configs. Finding local dev unit's config (most recent), checking python openvr availability.


Get-ChildItem "C:\Program Files (x86)\Steam\config\lighthouse" -Directory | Sort-Object LastWriteTime -Descending | Select-Object -First 6 Name, LastWriteTime; python -c "import openvr; print('openvr', openvr.__file__)" 2>&1; python --version

Find most-recent lighthouse configs, check pyopenvr


Python 3.12.0
Name         LastWriteTime
----         -------------
lhr-1f8e25f1 5/29/2026 3:28:27 AM
lhr-881e7037 3/13/2026 1:13:42 AM
lhr-86986b50 11/2/2025 4:47:51 AM
lhr-c4dd6b02 11/2/2025 1:18:51 AM
lhr-211d96a8 11/2/2025 1:15:57 AM
lhr-58e9d2d6 11/2/2025 1:15:54 AM
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'openvr'


Get-ChildItem "C:\Program Files (x86)\Steam\config\lighthouse\lhr-1f8e25f1" -Recurse | Select-Object FullName, Length

Inspect most-recent unit cache dir


FullName                                                                            Length
--------                                                                            ------
C:\Program Files (x86)\Steam\config\lighthouse\lhr-1f8e25f1\config.json              12254
C:\Program Files (x86)\Steam\config\lighthouse\lhr-1f8e25f1\eyetracking_runtime.log    183


$c = Get-Content "C:\Program Files (x86)\Steam\config\lighthouse\lhr-1f8e25f1\config.json" -Raw | ConvertFrom-Json; $c.manufacturer; $c.model_number; $c.device_serial_number; $c.tracking_to_eye_transform[0].distortion | ConvertTo-Json -Depth 5; $c.tracking_to_eye_transform[0].grow_for_undistort; $c.tracking_to_eye_transform[0].undistort_r2_cutoff; $c.tracking_to_eye_transform[0].intrinsics | ConvertTo-Json -Depth 3 -Compress

Check unit config distortion block


Bigscreen
Beyond
LHR-1F8E25F1
{
  "center_x": 0.0103014396355135,
  "center_y": -0.0117495571668924,
  "coeffs": [
    0.2438109586988374,
    0.0976338431275935,
    0.0808566995670431
  ],
  "type": "DISTORT_POLY3"
}
0.6
1
[[1.412994513938271,0.0,-0.0103014396355134],[0.0,1.4119978158893527,-0.0117495571668924],[0.0,0.0,-1.0]]


python -m pip install openvr 2>&1 | Select-Object -Last 3

Install pyopenvr package


[notice] A new release of pip is available: 26.0.1 -> 26.1.2
[notice] To update, run: python.exe -m pip install --upgrade pip


python -c "import openvr; print(openvr.__version__); import inspect; print(inspect.signature(openvr.IVRSystem.computeDistortion))"

Verify pyopenvr API signature


2.12.1401
(self, eye, u: float, v: float)


python -c "import openvr, inspect; src = inspect.getsource(openvr.IVRSystem.computeDistortion); print(src)"

Inspect computeDistortion wrapper source


    def computeDistortion(self, eye, u: float, v: float):
        """
        Gets the result of the distortion function for the specified eye and input UVs. UVs go from 0,0 in
        the upper left of that eye's viewport and 1,1 in the lower right of that eye's viewport.
        Returns true for success. Otherwise, returns false, and distortion coordinates are not suitable.
        """
        fn = self.function_table.computeDistortion
        distortionCoordinates = DistortionCoordinates_t()
        result = fn(eye, u, v, byref(distortionCoordinates))
        return result, distortionCoordinates


note that i am physically at enlyzeam (ssh `decid@enlyzeam`) where a different beyond is (it will have a different distortion profile). once your spike binaries are proven over there, will need to run the eyes-in test on enlyzeam


Writing spike S3 tooling.


Write

"""Standalone Beyond distortion evaluator — spike S3.

Maps panel/output UV -> pre-distortion render-target UV, per color channel,
from the device config JSON (tracking_to_eye_transform block). No SteamVR.

The open question this spike settles: production B1 configs use type
DISTORT_POLY3 with POSITIVE leading coeffs (+0.2438 on this unit), while
EVT/DVT used DISTORT_DPOLY3 with NEGATIVE leading coeffs (-0.22). Public
reimplementations (Monado u_compute_distortion_vive) evaluate the Vive
DPOLY3 family as a RECIPROCAL: d = 1/(1 + k1*r2 + k2*r4 + k3*r6).
Hypothesis: POLY3 is the same curve fitted the other way round, i.e.
d = 1 + k1*r2 + k2*r4 + k3*r6 directly. Four candidate forms are
implemented; verify.py picks the one matching OpenVR ComputeDistortion.

Hypothesis axes:
  form: 'poly'  -> d = 1 + k1*r2 + k2*r4 + k3*r6
        'recip' -> d = 1 / (1 + k1*r2 + k2*r4 + k3*r6)
  grow: True  -> output scale 0.5/(1+grow_for_undistort)   (Monado)
        False -> output scale 0.5                          (no grow rescale)
"""

import json

CHANNELS = ("distortion_red", "distortion", "distortion_blue")  # r, g, b
HYPOTHESES = [
    ("poly+grow", "poly", True),
    ("poly", "poly", False),
    ("recip+grow", "recip", True),
    ("recip", "recip", False),
]

class EyeDistortion:
    """Distortion model for one eye, built from tracking_to_eye_transform[eye]."""

    def __init__(self, tte):
        self.grow = tte["grow_for_undistort"]
        self.r2_cutoff = tte["undistort_r2_cutoff"]
        # center is shared across channels within an eye on the Beyond,
        # but read per channel anyway (B2 may differ).
        self.channels = []
        for key in CHANNELS:
            block = tte[key]
            self.channels.append({
                "cx": block["center_x"],
                "cy": block["center_y"],
                "k": list(block["coeffs"]),
                "type": block["type"],
            })

    def compute(self, u, v, form="poly", use_grow=True, aspect_x_over_y=1.0):
        """Panel UV (0..1, origin upper-left per OpenVR) -> source UV per channel.

        Returns ((ur, vr), (ug, vg), (ub, vb)).
        """
        scale = 0.5 / (1.0 + self.grow) if use_grow else 0.5
        out = []
        for ch in self.channels:
            tx = 2.0 * u - 1.0
            ty = (2.0 * v - 1.0) / aspect_x_over_y
            tx -= ch["cx"]
            ty -= ch["cy"]
            r2 = tx * tx + ty * ty
            if r2 > self.r2_cutoff:
                r2 = self.r2_cutoff
            k1, k2, k3 = ch["k"]
            p = 1.0 + r2 * (k1 + r2 * (k2 + r2 * k3))
            d = p if form == "poly" else 1.0 / p
            sx = 0.5 + (tx * d + ch["cx"]) * scale
            sy = 0.5 + (ty * d + ch["cy"]) * scale * aspect_x_over_y
            out.append((sx, sy))
        return tuple(out)

def load_config(path):
    """Load device config.json -> (left EyeDistortion, right EyeDistortion, raw dict)."""
    with open(path, "r") as f:
        cfg = json.load(f)
    tte = cfg["tracking_to_eye_transform"]
    return EyeDistortion(tte[0]), EyeDistortion(tte[1]), cfg


File created successfully at: C:\Users\decid\Documents\projects\sauna\spikes\s3-distortion-truth\poly3.py (file state is current in your context — no need to Read it back)

"""Standalone Beyond distortion evaluator — spike S3.

Maps panel/output UV -> pre-distortion render-target UV, per color channel,
from the device config JSON (tracking_to_eye_transform block). No SteamVR.

The open question this spike settles: production B1 configs use type
DISTORT_POLY3 with POSITIVE leading coeffs (+0.2438 on this unit), while
EVT/DVT used DISTORT_DPOLY3 with NEGATIVE leading coeffs (-0.22). Public
reimplementations (Monado u_compute_distortion_vive) evaluate the Vive
DPOLY3 family as a RECIPROCAL: d = 1/(1 + k1*r2 + k2*r4 + k3*r6).
Hypothesis: POLY3 is the same curve fitted the other way round, i.e.
d = 1 + k1*r2 + k2*r4 + k3*r6 directly. Four candidate forms are
implemented; verify.py picks the one matching OpenVR ComputeDistortion.

Hypothesis axes:
  form: 'poly'  -> d = 1 + k1*r2 + k2*r4 + k3*r6
        'recip' -> d = 1 / (1 + k1*r2 + k2*r4 + k3*r6)
  grow: True  -> output scale 0.5/(1+grow_for_undistort)   (Monado)
        False -> output scale 0.5                          (no grow rescale)
"""

import json

CHANNELS = ("distortion_red", "distortion", "distortion_blue")  # r, g, b
HYPOTHESES = [
    ("poly+grow", "poly", True),
    ("poly", "poly", False),
    ("recip+grow", "recip", True),
    ("recip", "recip", False),
]

class EyeDistortion:
    """Distortion model for one eye, built from tracking_to_eye_transform[eye]."""

    def __init__(self, tte):
        self.grow = tte["grow_for_undistort"]
        self.r2_cutoff = tte["undistort_r2_cutoff"]
        # center is shared across channels within an eye on the Beyond,
        # but read per channel anyway (B2 may differ).
        self.channels = []
        for key in CHANNELS:
            block = tte[key]
            self.channels.append({
                "cx": block["center_x"],
                "cy": block["center_y"],
                "k": list(block["coeffs"]),
                "type": block["type"],
            })

    def compute(self, u, v, form="poly", use_grow=True, aspect_x_over_y=1.0):
        """Panel UV (0..1, origin upper-left per OpenVR) -> source UV per channel.

        Returns ((ur, vr), (ug, vg), (ub, vb)).
        """
        scale = 0.5 / (1.0 + self.grow) if use_grow else 0.5
        out = []
        for ch in self.channels:
            tx = 2.0 * u - 1.0
            ty = (2.0 * v - 1.0) / aspect_x_over_y
            tx -= ch["cx"]
            ty -= ch["cy"]
            r2 = tx * tx + ty * ty
            if r2 > self.r2_cutoff:
                r2 = self.r2_cutoff
            k1, k2, k3 = ch["k"]
            p = 1.0 + r2 * (k1 + r2 * (k2 + r2 * k3))
            d = p if form == "poly" else 1.0 / p
            sx = 0.5 + (tx * d + ch["cx"]) * scale
            sy = 0.5 + (ty * d + ch["cy"]) * scale * aspect_x_over_y
            out.append((sx, sy))
        return tuple(out)

def load_config(path):
    """Load device config.json -> (left EyeDistortion, right EyeDistortion, raw dict)."""
    with open(path, "r") as f:
        cfg = json.load(f)
    tte = cfg["tracking_to_eye_transform"]
    return EyeDistortion(tte[0]), EyeDistortion(tte[1]), cfg


Prompt: note that i am physically at enlyzeam (ssh `decid@enlyzeam`) where a different beyond is (it will have a different distortion profile). once your spike binaries are proven over there, will need to run the eyes-in test on enlyzeam


Noted — ground-truth dump + eyes-in run on enlyzeam's Beyond, its own config. Will prove tooling locally first, then deploy over SSH. Continuing file writes.


"""Dump OpenVR ground truth for spike S3 — run ONCE per unit, dev-time only.

Requires SteamVR RUNNING with the Beyond connected (it will grab direct
mode; that's fine, this is the one sanctioned SteamVR run). Dumps:

  - ComputeDistortion UV grid per eye per channel (the ground truth)
  - GetProjectionRaw tangents per eye        (cross-check vs intrinsics)
  - GetEyeToHeadTransform per eye            (cross-check vs eye_to_head + ipd)
  - RecommendedRenderTargetSize, HMD serial/model

Output: distortion_dump_<serial>.json in this directory.

Usage:  python dump_openvr.py [--grid N]   (default 65 -> 65x65 per eye)
"""

import argparse
import json
import sys

import openvr

GRID_DEFAULT = 65

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--grid", type=int, default=GRID_DEFAULT)
    args = ap.parse_args()
    n = args.grid

    try:
        openvr.init(openvr.VRApplication_Background)
    except openvr.error_code.InitError as e:
        print(f"OpenVR init failed: {e}")
        print("Start SteamVR with the Beyond connected, then re-run.")
        sys.exit(1)

    vrsys = openvr.VRSystem()
    hmd = openvr.k_unTrackedDeviceIndex_Hmd
    serial = vrsys.getStringTrackedDeviceProperty(hmd, openvr.Prop_SerialNumber_String)
    model = vrsys.getStringTrackedDeviceProperty(hmd, openvr.Prop_ModelNumber_String)
    rt_w, rt_h = vrsys.getRecommendedRenderTargetSize()
    print(f"HMD: {model} {serial}, recommended RT {rt_w}x{rt_h} per eye")

    out = {
        "serial": serial,
        "model": model,
        "recommended_render_target": [rt_w, rt_h],
        "grid": n,
        "eyes": [],
    }

    for eye_name, eye in (("left", openvr.Eye_Left), ("right", openvr.Eye_Right)):
        left, right, top, bottom = vrsys.getProjectionRaw(eye)
        m = vrsys.getEyeToHeadTransform(eye)
        eye_to_head = [[m[r][c] for c in range(4)] for r in range(3)]

        points = []
        bad = 0
        for j in range(n):
            v = j / (n - 1)
            for i in range(n):
                u = i / (n - 1)
                ok, dc = vrsys.computeDistortion(eye, u, v)
                if not ok:
                    bad += 1
                    points.append(None)
                    continue
                points.append({
                    "uv": [u, v],
                    "r": [dc.rfRed[0], dc.rfRed[1]],
                    "g": [dc.rfGreen[0], dc.rfGreen[1]],
                    "b": [dc.rfBlue[0], dc.rfBlue[1]],
                })
        print(f"  {eye_name}: {n*n - bad}/{n*n} points ok, "
              f"projection_raw L{left:+.4f} R{right:+.4f} T{top:+.4f} B{bottom:+.4f}")
        out["eyes"].append({
            "eye": eye_name,
            "projection_raw": [left, right, top, bottom],
            "eye_to_head": eye_to_head,
            "points": points,
        })

    openvr.shutdown()

    fname = f"distortion_dump_{serial.lower().replace('-', '_')}.json"
    with open(fname, "w") as f:
        json.dump(out, f)
    print(f"Wrote {fname}")

if __name__ == "__main__":
    main()


File created successfully at: C:\Users\decid\Documents\projects\sauna\spikes\s3-distortion-truth\dump_openvr.py (file state is current in your context — no need to Read it back)

"""Dump OpenVR ground truth for spike S3 — run ONCE per unit, dev-time only.

Requires SteamVR RUNNING with the Beyond connected (it will grab direct
mode; that's fine, this is the one sanctioned SteamVR run). Dumps:

  - ComputeDistortion UV grid per eye per channel (the ground truth)
  - GetProjectionRaw tangents per eye        (cross-check vs intrinsics)
  - GetEyeToHeadTransform per eye            (cross-check vs eye_to_head + ipd)
  - RecommendedRenderTargetSize, HMD serial/model

Output: distortion_dump_<serial>.json in this directory.

Usage:  python dump_openvr.py [--grid N]   (default 65 -> 65x65 per eye)
"""

import argparse
import json
import sys

import openvr

GRID_DEFAULT = 65

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--grid", type=int, default=GRID_DEFAULT)
    args = ap.parse_args()
    n = args.grid

    try:
        openvr.init(openvr.VRApplication_Background)
    except openvr.error_code.InitError as e:
        print(f"OpenVR init failed: {e}")
        print("Start SteamVR with the Beyond connected, then re-run.")
        sys.exit(1)

    vrsys = openvr.VRSystem()
    hmd = openvr.k_unTrackedDeviceIndex_Hmd
    serial = vrsys.getStringTrackedDeviceProperty(hmd, openvr.Prop_SerialNumber_String)
    model = vrsys.getStringTrackedDeviceProperty(hmd, openvr.Prop_ModelNumber_String)
    rt_w, rt_h = vrsys.getRecommendedRenderTargetSize()
    print(f"HMD: {model} {serial}, recommended RT {rt_w}x{rt_h} per eye")

    out = {
        "serial": serial,
        "model": model,
        "recommended_render_target": [rt_w, rt_h],
        "grid": n,
        "eyes": [],
    }

    for eye_name, eye in (("left", openvr.Eye_Left), ("right", openvr.Eye_Right)):
        left, right, top, bottom = vrsys.getProjectionRaw(eye)
        m = vrsys.getEyeToHeadTransform(eye)
        eye_to_head = [[m[r][c] for c in range(4)] for r in range(3)]

        points = []
        bad = 0
        for j in range(n):
            v = j / (n - 1)
            for i in range(n):
                u = i / (n - 1)
                ok, dc = vrsys.computeDistortion(eye, u, v)
                if not ok:
                    bad += 1
                    points.append(None)
                    continue
                points.append({
                    "uv": [u, v],
                    "r": [dc.rfRed[0], dc.rfRed[1]],
                    "g": [dc.rfGreen[0], dc.rfGreen[1]],
                    "b": [dc.rfBlue[0], dc.rfBlue[1]],
                })
        print(f"  {eye_name}: {n*n - bad}/{n*n} points ok, "
              f"projection_raw L{left:+.4f} R{right:+.4f} T{top:+.4f} B{bottom:+.4f}")
        out["eyes"].append({
            "eye": eye_name,
            "projection_raw": [left, right, top, bottom],
            "eye_to_head": eye_to_head,
            "points": points,
        })

    openvr.shutdown()

    fname = f"distortion_dump_{serial.lower().replace('-', '_')}.json"
    with open(fname, "w") as f:
        json.dump(out, f)
    print(f"Wrote {fname}")

if __name__ == "__main__":
    main()


"""Spike S3 verdict: test each evaluator hypothesis against the OpenVR dump.

Usage:
  python verify.py <distortion_dump_*.json> <config.json>
  python verify.py --selftest

Error metric: source-UV delta scaled to panel pixels (x2544) — the units the
warp actually renders in. Pass criterion (PLAN.md): sub-pixel, i.e. winning
hypothesis max error < 1.0 px across both eyes, all three channels.

Also cross-checks projection_raw vs config intrinsics and eye_to_head
rotation vs config (canting), since the dump carries them for free.
"""

import json
import math
import sys

from poly3 import EyeDistortion, HYPOTHESES, load_config

PANEL_PX = 2544  # per-eye panel pixels (eye_target_width_in_pixels)

def test_hypothesis(dump, eyes_cfg, form, use_grow):
    """Return dict of per-channel (rms_px, max_px) plus overall max."""
    stats = {}
    overall_max = 0.0
    for eye_idx, eye_dump in enumerate(dump["eyes"]):
        model = eyes_cfg[eye_idx]
        for ch_idx, ch_name in enumerate(("r", "g", "b")):
            se = 0.0
            mx = 0.0
            count = 0
            for pt in eye_dump["points"]:
                if pt is None:
                    continue
                u, v = pt["uv"]
                ours = model.compute(u, v, form=form, use_grow=use_grow)[ch_idx]
                theirs = pt[ch_name]
                dx = (ours[0] - theirs[0]) * PANEL_PX
                dy = (ours[1] - theirs[1]) * PANEL_PX
                e2 = dx * dx + dy * dy
                se += e2
                if e2 > mx:
                    mx = e2
                count += 1
            rms = math.sqrt(se / count) if count else float("nan")
            mxe = math.sqrt(mx)
            stats[(eye_dump["eye"], ch_name)] = (rms, mxe)
            overall_max = max(overall_max, mxe)
    stats["overall_max"] = overall_max
    return stats

def cross_checks(dump, cfg):
    print("\nCross-checks (dump vs config):")
    for eye_idx, eye_dump in enumerate(dump["eyes"]):
        tte = cfg["tracking_to_eye_transform"][eye_idx]
        K = tte["intrinsics"]
        fx, cx = K[0][0], -K[0][2]
        fy, cy = K[1][1], -K[1][2]
        # OpenVR raw projection: tangents left/right/top/bottom.
        # From intrinsics: left = (-1 - cx)/fx ... (Valve convention cy sign
        # varies; report both so the dump settles it.)
        exp_l, exp_r = (-1 - cx) / fx, (1 - cx) / fx
        exp_t, exp_b = (-1 - cy) / fy, (1 - cy) / fy
        L, R, T, B = eye_dump["projection_raw"]
        print(f"  {eye_dump['eye']} projection_raw  dump L{L:+.5f} R{R:+.5f} T{T:+.5f} B{B:+.5f}")
        print(f"  {eye_dump['eye']} from intrinsics      L{exp_l:+.5f} R{exp_r:+.5f} T{exp_t:+.5f} B{exp_b:+.5f}")
        e2h_cfg = tte["eye_to_head"]
        e2h_dump = eye_dump["eye_to_head"]
        rot_err = max(
            abs(e2h_cfg[r][c] - e2h_dump[r][c]) for r in range(3) for c in range(3)
        )
        tx = e2h_dump[0][3] * 1000.0
        print(f"  {eye_dump['eye']} eye_to_head rot max-abs-diff {rot_err:.2e}; "
              f"translation x {tx:+.3f} mm (ipd.default_mm/2 expected)")
    ipd = cfg.get("ipd", {})
    print(f"  config ipd block: {ipd}")

def selftest():
    """Synthetic round trip: generate a fake dump from a known hypothesis and
    confirm verify picks exactly that hypothesis with ~0 error."""
    tte = {
        "grow_for_undistort": 0.6,
        "undistort_r2_cutoff": 1.0,
        "distortion": {"center_x": 0.01, "center_y": -0.012,
                       "coeffs": [0.2438, 0.0976, 0.0808], "type": "DISTORT_POLY3"},
        "distortion_red": {"center_x": 0.01, "center_y": -0.012,
                           "coeffs": [0.2159, 0.0887, 0.1174], "type": "DISTORT_POLY3"},
        "distortion_blue": {"center_x": 0.01, "center_y": -0.012,
                            "coeffs": [0.2805, -0.0059, 0.1701], "type": "DISTORT_POLY3"},
    }
    cfg = {"tracking_to_eye_transform": [tte, tte]}
    eyes = [EyeDistortion(tte), EyeDistortion(tte)]
    truth_form, truth_grow = "recip", True  # pretend Valve uses Monado form

    n = 17
    dump = {"eyes": []}
    for eye_idx, name in enumerate(("left", "right")):
        points = []
        for j in range(n):
            for i in range(n):
                u, v = i / (n - 1), j / (n - 1)
                r, g, b = eyes[eye_idx].compute(u, v, form=truth_form, use_grow=truth_grow)
                points.append({"uv": [u, v], "r": list(r), "g": list(g), "b": list(b)})
        dump["eyes"].append({"eye": name, "points": points})

    results = {name: test_hypothesis(dump, eyes, form, grow)
               for name, form, grow in HYPOTHESES}
    report(results)
    winner = min(results, key=lambda k: results[k]["overall_max"])
    assert winner == "recip+grow", f"selftest failed: winner {winner}"
    assert results[winner]["overall_max"] < 1e-9
    # All other hypotheses must be clearly distinguishable (not sub-pixel).
    for name, st in results.items():
        if name != winner:
            assert st["overall_max"] > 1.0, f"hypothesis {name} not distinguishable"
    print("\nSELFTEST PASS — winner recip+grow as constructed, others distinguishable")

def report(results):
    print(f"\n{'hypothesis':<12} {'overall max px':>14}   per-channel rms px (L/R averaged)")
    for name, st in results.items():
        per_ch = []
        for ch in ("r", "g", "b"):
            vals = [st[k][0] for k in st if isinstance(k, tuple) and k[1] == ch]
            per_ch.append(f"{ch}:{sum(vals)/len(vals):8.3f}")
        print(f"{name:<12} {st['overall_max']:>14.4f}   {' '.join(per_ch)}")

def main():
    if "--selftest" in sys.argv:
        selftest()
        return
    if len(sys.argv) != 3:
        print(__doc__)
        sys.exit(2)

    with open(sys.argv[1]) as f:
        dump = json.load(f)
    left, right, cfg = load_config(sys.argv[2])
    eyes_cfg = [left, right]

    serial_cfg = cfg.get("device_serial_number", "?")
    print(f"dump unit: {dump.get('serial')}   config unit: {serial_cfg}")
    if dump.get("serial", "").lower() != serial_cfg.lower():
        print("WARNING: serial mismatch — dump and config are from different units!")

    results = {name: test_hypothesis(dump, eyes_cfg, form, grow)
               for name, form, grow in HYPOTHESES}
    report(results)

    winner = min(results, key=lambda k: results[k]["overall_max"])
    wmax = results[winner]["overall_max"]
    verdict = "PASS" if wmax < 1.0 else "FAIL"
    print(f"\nWinner: {winner}  (overall max {wmax:.4f} px)  ->  {verdict} "
          f"(criterion: sub-pixel)")

    cross_checks(dump, cfg)

if __name__ == "__main__":
    main()


File created successfully at: C:\Users\decid\Documents\projects\sauna\spikes\s3-distortion-truth\verify.py (file state is current in your context — no need to Read it back)

"""Spike S3 verdict: test each evaluator hypothesis against the OpenVR dump.

Usage:
  python verify.py <distortion_dump_*.json> <config.json>
  python verify.py --selftest

Error metric: source-UV delta scaled to panel pixels (x2544) — the units the
warp actually renders in. Pass criterion (PLAN.md): sub-pixel, i.e. winning
hypothesis max error < 1.0 px across both eyes, all three channels.

Also cross-checks projection_raw vs config intrinsics and eye_to_head
rotation vs config (canting), since the dump carries them for free.
"""

import json
import math
import sys

from poly3 import EyeDistortion, HYPOTHESES, load_config

PANEL_PX = 2544  # per-eye panel pixels (eye_target_width_in_pixels)

def test_hypothesis(dump, eyes_cfg, form, use_grow):
    """Return dict of per-channel (rms_px, max_px) plus overall max."""
    stats = {}
    overall_max = 0.0
    for eye_idx, eye_dump in enumerate(dump["eyes"]):
        model = eyes_cfg[eye_idx]
        for ch_idx, ch_name in enumerate(("r", "g", "b")):
            se = 0.0
            mx = 0.0
            count = 0
            for pt in eye_dump["points"]:
                if pt is None:
                    continue
                u, v = pt["uv"]
                ours = model.compute(u, v, form=form, use_grow=use_grow)[ch_idx]
                theirs = pt[ch_name]
                dx = (ours[0] - theirs[0]) * PANEL_PX
                dy = (ours[1] - theirs[1]) * PANEL_PX
                e2 = dx * dx + dy * dy
                se += e2
                if e2 > mx:
                    mx = e2
                count += 1
            rms = math.sqrt(se / count) if count else float("nan")
            mxe = math.sqrt(mx)
            stats[(eye_dump["eye"], ch_name)] = (rms, mxe)
            overall_max = max(overall_max, mxe)
    stats["overall_max"] = overall_max
    return stats

def cross_checks(dump, cfg):
    print("\nCross-checks (dump vs config):")
    for eye_idx, eye_dump in enumerate(dump["eyes"]):
        tte = cfg["tracking_to_eye_transform"][eye_idx]
        K = tte["intrinsics"]
        fx, cx = K[0][0], -K[0][2]
        fy, cy = K[1][1], -K[1][2]
        # OpenVR raw projection: tangents left/right/top/bottom.
        # From intrinsics: left = (-1 - cx)/fx ... (Valve convention cy sign
        # varies; report both so the dump settles it.)
        exp_l, exp_r = (-1 - cx) / fx, (1 - cx) / fx
        exp_t, exp_b = (-1 - cy) / fy, (1 - cy) / fy
        L, R, T, B = eye_dump["projection_raw"]
        print(f"  {eye_dump['eye']} projection_raw  dump L{L:+.5f} R{R:+.5f} T{T:+.5f} B{B:+.5f}")
        print(f"  {eye_dump['eye']} from intrinsics      L{exp_l:+.5f} R{exp_r:+.5f} T{exp_t:+.5f} B{exp_b:+.5f}")
        e2h_cfg = tte["eye_to_head"]
        e2h_dump = eye_dump["eye_to_head"]
        rot_err = max(
            abs(e2h_cfg[r][c] - e2h_dump[r][c]) for r in range(3) for c in range(3)
        )
        tx = e2h_dump[0][3] * 1000.0
        print(f"  {eye_dump['eye']} eye_to_head rot max-abs-diff {rot_err:.2e}; "
              f"translation x {tx:+.3f} mm (ipd.default_mm/2 expected)")
    ipd = cfg.get("ipd", {})
    print(f"  config ipd block: {ipd}")

def selftest():
    """Synthetic round trip: generate a fake dump from a known hypothesis and
    confirm verify picks exactly that hypothesis with ~0 error."""
    tte = {
        "grow_for_undistort": 0.6,
        "undistort_r2_cutoff": 1.0,
        "distortion": {"center_x": 0.01, "center_y": -0.012,
                       "coeffs": [0.2438, 0.0976, 0.0808], "type": "DISTORT_POLY3"},
        "distortion_red": {"center_x": 0.01, "center_y": -0.012,
                           "coeffs": [0.2159, 0.0887, 0.1174], "type": "DISTORT_POLY3"},
        "distortion_blue": {"center_x": 0.01, "center_y": -0.012,
                            "coeffs": [0.2805, -0.0059, 0.1701], "type": "DISTORT_POLY3"},
    }
    cfg = {"tracking_to_eye_transform": [tte, tte]}
    eyes = [EyeDistortion(tte), EyeDistortion(tte)]
    truth_form, truth_grow = "recip", True  # pretend Valve uses Monado form

    n = 17
    dump = {"eyes": []}
    for eye_idx, name in enumerate(("left", "right")):
        points = []
        for j in range(n):
            for i in range(n):
                u, v = i / (n - 1), j / (n - 1)
                r, g, b = eyes[eye_idx].compute(u, v, form=truth_form, use_grow=truth_grow)
                points.append({"uv": [u, v], "r": list(r), "g": list(g), "b": list(b)})
        dump["eyes"].append({"eye": name, "points": points})

    results = {name: test_hypothesis(dump, eyes, form, grow)
               for name, form, grow in HYPOTHESES}
    report(results)
    winner = min(results, key=lambda k: results[k]["overall_max"])
    assert winner == "recip+grow", f"selftest failed: winner {winner}"
    assert results[winner]["overall_max"] < 1e-9
    # All other hypotheses must be clearly distinguishable (not sub-pixel).
    for name, st in results.items():
        if name != winner:
            assert st["overall_max"] > 1.0, f"hypothesis {name} not distinguishable"
    print("\nSELFTEST PASS — winner recip+grow as constructed, others distinguishable")

def report(results):
    print(f"\n{'hypothesis':<12} {'overall max px':>14}   per-channel rms px (L/R averaged)")
    for name, st in results.items():
        per_ch = []
        for ch in ("r", "g", "b"):
            vals = [st[k][0] for k in st if isinstance(k, tuple) and k[1] == ch]
            per_ch.append(f"{ch}:{sum(vals)/len(vals):8.3f}")
        print(f"{name:<12} {st['overall_max']:>14.4f}   {' '.join(per_ch)}")

def main():
    if "--selftest" in sys.argv:
        selftest()
        return
    if len(sys.argv) != 3:
        print(__doc__)
        sys.exit(2)

    with open(sys.argv[1]) as f:
        dump = json.load(f)
    left, right, cfg = load_config(sys.argv[2])
    eyes_cfg = [left, right]

    serial_cfg = cfg.get("device_serial_number", "?")
    print(f"dump unit: {dump.get('serial')}   config unit: {serial_cfg}")
    if dump.get("serial", "").lower() != serial_cfg.lower():
        print("WARNING: serial mismatch — dump and config are from different units!")

    results = {name: test_hypothesis(dump, eyes_cfg, form, grow)
               for name, form, grow in HYPOTHESES}
    report(results)

    winner = min(results, key=lambda k: results[k]["overall_max"])
    wmax = results[winner]["overall_max"]
    verdict = "PASS" if wmax < 1.0 else "FAIL"
    print(f"\nWinner: {winner}  (overall max {wmax:.4f} px)  ->  {verdict} "
          f"(criterion: sub-pixel)")

    cross_checks(dump, cfg)

if __name__ == "__main__":
    main()


PowerShell

cd C:\Users\decid\Documents\projects\sauna\spikes\s3-distortion-truth; python verify.py --selftest

Run evaluator selftest


hypothesis   overall max px   per-channel rms px (L/R averaged)
poly+grow          855.3592   r: 421.789 g: 424.695 b: 442.856
poly              1833.2245   r: 969.135 g: 974.094 b: 998.962
recip+grow           0.0000   r:   0.000 g:   0.000 b:   0.000
recip              476.5902   r: 313.650 g: 312.419 b: 309.176

SELFTEST PASS � winner recip+grow as constructed, others distinguishable


Selftest PASS — hypotheses distinguishable by hundreds of px, winner exact. Writing README + FINDINGS skeleton, then checking enlyzeam.


Write

# Spike S3 — distortion ground truth

**Question (PLAN.md):** which direction do the production `DISTORT_POLY3` coeffs run?
EVT/DVT configs used `DISTORT_DPOLY3` with negative leading coeffs (−0.22); production B1
uses `DISTORT_POLY3` with positive ones (+0.24). Public reimplementations (Monado) evaluate
the DPOLY3 family as `d = 1/(1 + k1·r² + k2·r⁴ + k3·r⁶)`. Is POLY3 the same curve fitted the
other way (`d = 1 + …` direct), and is the `0.5/(1+grow_for_undistort)` output rescale right?

**Pass criterion:** standalone evaluator matches an OpenVR `ComputeDistortion` UV-grid dump
from one dev unit within sub-pixel tolerance (max error < 1 px at 2544 px/eye).

## Files

| file | role |
|---|---|
| `poly3.py` | standalone evaluator, 4 hypothesis variants (form × grow-rescale) |
| `dump_openvr.py` | ground-truth dump — needs SteamVR running with the Beyond (one sanctioned dev-time run) |
| `verify.py` | tests all hypotheses against a dump; `--selftest` for synthetic round-trip |

## Run order (on the rig with the headset)

```
pip install openvr
# 1. Start SteamVR with the Beyond connected (it takes direct mode — fine).
python dump_openvr.py                # writes distortion_dump_<serial>.json
# 2. Quit SteamVR.
python verify.py distortion_dump_<serial>.json "C:\Program Files (x86)\Steam\config\lighthouse\<lhr-serial>\config.json"
```

The config path is SteamVR's cache of the unit's SIP flash JSON (refreshed by the same
SteamVR run). `verify.py` warns if dump and config serials differ.

Bonus ground truth captured in the same dump: `GetProjectionRaw` (settles intrinsics→frustum
convention incl. cy sign), `GetEyeToHeadTransform` (settles canting matrix + which side gets
+ipd/2) — both cross-checked by `verify.py` for M2.


File created successfully at: C:\Users\decid\Documents\projects\sauna\spikes\s3-distortion-truth\README.md (file state is current in your context — no need to Read it back)

# Spike S3 — distortion ground truth

**Question (PLAN.md):** which direction do the production `DISTORT_POLY3` coeffs run?
EVT/DVT configs used `DISTORT_DPOLY3` with negative leading coeffs (−0.22); production B1
uses `DISTORT_POLY3` with positive ones (+0.24). Public reimplementations (Monado) evaluate
the DPOLY3 family as `d = 1/(1 + k1·r² + k2·r⁴ + k3·r⁶)`. Is POLY3 the same curve fitted the
other way (`d = 1 + …` direct), and is the `0.5/(1+grow_for_undistort)` output rescale right?

**Pass criterion:** standalone evaluator matches an OpenVR `ComputeDistortion` UV-grid dump
from one dev unit within sub-pixel tolerance (max error < 1 px at 2544 px/eye).

## Files

| file | role |
|---|---|
| `poly3.py` | standalone evaluator, 4 hypothesis variants (form × grow-rescale) |
| `dump_openvr.py` | ground-truth dump — needs SteamVR running with the Beyond (one sanctioned dev-time run) |
| `verify.py` | tests all hypotheses against a dump; `--selftest` for synthetic round-trip |

## Run order (on the rig with the headset)

```
pip install openvr
# 1. Start SteamVR with the Beyond connected (it takes direct mode — fine).
python dump_openvr.py                # writes distortion_dump_<serial>.json
# 2. Quit SteamVR.
python verify.py distortion_dump_<serial>.json "C:\Program Files (x86)\Steam\config\lighthouse\<lhr-serial>\config.json"
```

The config path is SteamVR's cache of the unit's SIP flash JSON (refreshed by the same
SteamVR run). `verify.py` warns if dump and config serials differ.

Bonus ground truth captured in the same dump: `GetProjectionRaw` (settles intrinsics→frustum
convention incl. cy sign), `GetEyeToHeadTransform` (settles canting matrix + which side gets
+ipd/2) — both cross-checked by `verify.py` for M2.


PowerShell

ssh decid@enlyzeam "python --version; python -c \"import openvr; print('openvr ok')\" 2>&1; if exist \"C:\Program Files (x86)\Steam\steamapps\common\SteamVR\bin\win64\vrserver.exe\" (echo steamvr present) else (echo steamvr MISSING); dir /b \"C:\Program Files (x86)\Steam\config\lighthouse\" 2>nul"

Check python/openvr/SteamVR on enlyzeam


Error: Exit code 1
Python 3.9.5
  File "<string>", line 1
    \
     ^
SyntaxError: unexpected EOF while parsing
x86: The term 'x86' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.


Remote shell = PowerShell, quoting mangled. Retry with safe quoting.


ssh decid@enlyzeam "python -c ""import openvr; print('openvr', openvr.__version__)"""; ssh decid@enlyzeam "Test-Path 'C:\Program Files (x86)\Steam\steamapps\common\SteamVR\bin\win64\vrserver.exe'"; ssh decid@enlyzeam "Get-ChildItem 'C:\Program Files (x86)\Steam\config\lighthouse' -Directory | Sort-Object LastWriteTime -Descending | Select-Object -First 5 Name, LastWriteTime"

Check openvr, SteamVR, lighthouse cache on enlyzeam


Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'openvr'
True

Name         LastWriteTime
----         -------------
lhr-ff449365 4/27/2026 4:15:17 PM
lhr-fe89dcaa 4/27/2026 4:15:17 PM
lhr-fdf71e37 4/27/2026 4:15:17 PM
lhr-fb8bc0af 4/27/2026 4:15:17 PM
lhr-fb338ec0 4/27/2026 4:15:17 PM


ssh decid@enlyzeam "python -m pip install openvr 2>&1 | Select-Object -Last 2; New-Item -ItemType Directory -Force C:\Users\decid\sauna-spikes\s3-distortion-truth | Out-Null"

Install openvr and create spike dir on enlyzeam


[notice] A new release of pip is available: 24.1 -> 26.0.1
[notice] To update, run: python.exe -m pip install --upgrade pip


Deploy spike files to enlyzeam and run selftest there


ssh decid@enlyzeam "python -c ""import openvr; print('openvr', openvr.__version__)""; cd C:\Users\decid\sauna-spikes\s3-distortion-truth; python dump_openvr.py"

Check openvr and attempt dump on enlyzeam


Error: Exit code 1
openvr 2.12.1401
OpenVR init failed:
Start SteamVR with the Beyond connected, then re-run.


git -C C:\Users\decid\Documents\projects\sauna status --short; git -C C:\Users\decid\Documents\projects\sauna log --oneline -1

Check git status


?? spikes/s3-distortion-truth/
52ec7de Spike S2 PASS: own-process direct-mode present, visually validated on two rigs


git -C C:\Users\decid\Documents\projects\sauna add spikes/s3-distortion-truth && git -C C:\Users\decid\Documents\projects\sauna commit -m @'
Spike S3 scaffold: distortion evaluator + OpenVR ground-truth dump tooling

poly3.py: standalone panel-UV -> source-UV evaluator with 4 hypothesis
variants (poly vs reciprocal form x grow_for_undistort rescale on/off).
dump_openvr.py: one-shot SteamVR ComputeDistortion grid dump per eye per
channel, plus GetProjectionRaw and GetEyeToHeadTransform for M2 cross-checks.
verify.py: scores all hypotheses against a dump in panel pixels; synthetic
selftest proves hypotheses are distinguishable (hundreds of px apart).

Selftest passing on both rigs; deployed to enlyzeam sauna-spikes. Awaiting
the one sanctioned SteamVR run on the enlyzeam Beyond for ground truth.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
'@

Commit S3 scaffold


warning: in the working copy of 'spikes/s3-distortion-truth/README.md', LF will be replaced by CRLF the next time Git touches it
warning: in the working copy of 'spikes/s3-distortion-truth/dump_openvr.py', LF will be replaced by CRLF the next time Git touches it
warning: in the working copy of 'spikes/s3-distortion-truth/poly3.py', LF will be replaced by CRLF the next time Git touches it
warning: in the working copy of 'spikes/s3-distortion-truth/verify.py', LF will be replaced by CRLF the next time Git touches it
[main 4002b15] Spike S3 scaffold: distortion evaluator + OpenVR ground-truth dump tooling
 4 files changed, 373 insertions(+)
 create mode 100644 spikes/s3-distortion-truth/README.md
 create mode 100644 spikes/s3-distortion-truth/dump_openvr.py
 create mode 100644 spikes/s3-distortion-truth/poly3.py
 create mode 100644 spikes/s3-distortion-truth/verify.py


git -C C:\Users\decid\Documents\projects\sauna push

Push S3 scaffold commit


To https://github.com/FEAT-HER3/sauna.git
   52ec7de..4002b15  main -> main


S3 tooling built, proven on both rigs, committed + pushed (4002b15). Blocked now on one physical step only — you, at enlyzeam.

**What exists** (`spikes/s3-distortion-truth/`, mirrored to `enlyzeam C:\Users\decid\sauna-spikes\s3-distortion-truth\`):
- `poly3.py` — standalone evaluator, 4 hypotheses: {poly form `1+k1r²+…` vs Monado reciprocal `1/(1+…)`} × {grow rescale on/off}
- `dump_openvr.py` — one-shot SteamVR `ComputeDistortion` 65×65 grid per eye per channel + `GetProjectionRaw` + `GetEyeToHeadTransform` (free M2 ground truth: cy sign convention, canting matrix, which eye gets +ipd/2)
- `verify.py` — scores hypotheses in panel px; selftest PASS both rigs, hypotheses separated by hundreds of px so winner unambiguous

**Your step on enlyzeam:**
1. Beyond connected, start SteamVR (it grabs direct mode — that's the one sanctioned run).
2. Tell me when up — I run dump + verify over SSH. Fallback if cross-session OpenVR IPC refuses: you run locally in terminal there:
```
cd C:\Users\decid\sauna-spikes\s3-distortion-truth
python dump_openvr.py
python verify.py distortion_dump_<serial>.json "C:\Program Files (x86)\Steam\config\lighthouse\<lhr-serial>\config.json"
```
3. Quit SteamVR after dump.

`verify.py` warns on dump/config serial mismatch, so enlyzeam unit's distinct profile handled automatically (its cache dir refreshes during the run).

**Eyes-in test after:** once winner hypothesis confirmed numerically, plan = bake winning warp into s2_nvapi presenter path with a structure-revealing grid through the lens — straight lines should look straight eyes-in. That's the visual close-out on enlyzeam, then S3 FINDINGS.md + PLAN update.


beyond connected + steamvr up on enlyzeam


<owl_reachability_notice>
You have an active owl perch. When spawning subagents or doing work, use run_in_background: true so you stay reachable for incoming messages.
</owl_reachability_notice>


CAVEMAN MODE ACTIVE (full). Drop articles/filler/pleasantries/hedging. Fragments OK. Code/commits/security: write normal.


Running dump over SSH.


ssh decid@enlyzeam "cd C:\Users\decid\sauna-spikes\s3-distortion-truth; python dump_openvr.py"

Run OpenVR distortion dump on enlyzeam


Error: Exit code 1
OpenVR init failed:
Start SteamVR with the Beyond connected, then re-run.


Cross-session IPC blocked as suspected. Trying scheduled-task workaround to run in your interactive session.


ssh decid@enlyzeam "schtasks /create /f /tn saunaS3dump /tr 'cmd /c cd /d C:\Users\decid\sauna-spikes\s3-distortion-truth && python dump_openvr.py > dump_log.txt 2>&1' /sc once /st 23:59 /it; schtasks /run /tn saunaS3dump"

Create and run interactive-session dump task on enlyzeam


SUCCESS: The scheduled task "saunaS3dump" has successfully been created.
SUCCESS: Attempted to run the scheduled task "saunaS3dump".


Start-Sleep -Seconds 8; ssh decid@enlyzeam "cd C:\Users\decid\sauna-spikes\s3-distortion-truth; Get-Content dump_log.txt; Get-ChildItem distortion_dump_*.json | Select-Object Name, Length"

Check dump task output


HMD: Beyond LHR-599F3B91, recommended RT 3104x3104 per eye
  left: 4225/4225 points ok, projection_raw L-1.4000 R+0.9130 T-1.3798 B+0.9233
  right: 4225/4225 points ok, projection_raw L-0.9044 R+1.4020 T-1.3411 B+0.9411
Wrote distortion_dump_lhr_599f3b91.json

Name                               Length
----                               ------
distortion_dump_lhr_599f3b91.json 1440515


Dump OK via task trick — enlyzeam unit = LHR-599F3B91, 4225/4225 points both eyes. Running verify + cleaning up task.


ssh decid@enlyzeam "schtasks /delete /f /tn saunaS3dump | Out-Null; cd C:\Users\decid\sauna-spikes\s3-distortion-truth; python verify.py distortion_dump_lhr_599f3b91.json 'C:\Program Files (x86)\Steam\config\lighthouse\lhr-599f3b91\config.json'"

Run hypothesis verification on enlyzeam


dump unit: LHR-599F3B91   config unit: LHR-599F3B91

hypothesis   overall max px   per-channel rms px (L/R averaged)
poly+grow         1395.7371   r: 150.535 g: 157.060 b: 168.829
poly               686.7507   r: 451.670 g: 452.940 b: 455.401
recip+grow        2202.6093   r: 472.659 g: 484.309 b: 505.258
recip             1728.7205   r: 267.943 g: 278.061 b: 296.869

Winner: poly  (overall max 686.7507 px)  ->  FAIL (criterion: sub-pixel)

Cross-checks (dump vs config):
  left projection_raw  dump L-1.39998 R+0.91304 T-1.37976 B+0.92332
  left from intrinsics      L-0.69784 R+0.71157 T-0.68506 B+0.72266
  left eye_to_head rot max-abs-diff 1.07e-01; translation x -31.000 mm (ipd.default_mm/2 expected)
  right projection_raw  dump L-0.90444 R+1.40201 T-1.34110 B+0.94106
  right from intrinsics      L-0.70222 R+0.70144 T-0.68771 B+0.71415
  right eye_to_head rot max-abs-diff 1.07e-01; translation x +31.000 mm (ipd.default_mm/2 expected)
  config ipd block: {'default_mm': 69}


scp decid@enlyzeam:C:/Users/decid/sauna-spikes/s3-distortion-truth/distortion_dump_lhr_599f3b91.json C:\Users\decid\Documents\projects\sauna\spikes\s3-distortion-truth\; scp "decid@enlyzeam:\"C:/Program Files (x86)/Steam/config/lighthouse/lhr-599f3b91/config.json\"" C:\Users\decid\Documents\projects\sauna\spikes\s3-distortion-truth\config_lhr_599f3b91.json; python -c "import numpy; print(numpy.__version__)"

Fetch dump and config locally, check numpy


x86: The term 'x86' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
2.2.2


ssh decid@enlyzeam "Get-Content 'C:\Program Files (x86)\Steam\config\lighthouse\lhr-599f3b91\config.json' -Raw" | Set-Content C:\Users\decid\Documents\projects\sauna\spikes\s3-distortion-truth\config_lhr_599f3b91.json; Get-ChildItem C:\Users\decid\Documents\projects\sauna\spikes\s3-distortion-truth\*.json | Select-Object Name, Length

Fetch config via ssh cat


Name                               Length
----                               ------
config_lhr_599f3b91.json            12228
distortion_dump_lhr_599f3b91.json 1440515


Both files local. Inspecting enlyzeam unit's config blocks, then writing empirical fit.


python -c "
import json
cfg = json.load(open(r'C:\Users\decid\Documents\projects\sauna\spikes\s3-distortion-truth\config_lhr_599f3b91.json'))
for i, name in ((0,'LEFT'),(1,'RIGHT')):
    t = cfg['tracking_to_eye_transform'][i]
    print(name)
    print('  intrinsics', t['intrinsics'])
    print('  eye_to_head', t['eye_to_head'])
    print('  grow', t['grow_for_undistort'], 'cutoff', t['undistort_r2_cutoff'])
    for ch in ('distortion','distortion_red','distortion_blue'):
        b = t[ch]
        print(' ', ch, b['type'], 'center', round(b['center_x'],5), round(b['center_y'],5), 'coeffs', [round(c,5) for c in b['coeffs']])
print('ipd', cfg.get('ipd'))
"; python -c "import scipy; print('scipy', scipy.__version__)" 2>&1

Show enlyzeam unit config optics blocks


Exit code 1
LEFT
  intrinsics [[1.4190387502680653, 0.0, 0.0097386872668569], [0.0, 1.420734589979133, 0.026714069319401], [0.0, 0.0, -1.0]]
  eye_to_head [[0.9942073760240819, 0.0, 0.10747880469799379], [-0.011514392898854297, 0.9942448326331053, 0.10651118034525311], [-0.10686024618856305, -0.10713175431387381, 0.9884855461776623]]
  grow 0.6 cutoff 1
  distortion DISTORT_POLY3 center -0.00974 0.02671 coeffs [0.20891, 0.13323, 0.05599]
  distortion_red DISTORT_POLY3 center -0.00974 0.02671 coeffs [0.18585, 0.16702, 0.03718]
  distortion_blue DISTORT_POLY3 center -0.00974 0.02671 coeffs [0.25042, 0.07241, 0.08984]
RIGHT
  intrinsics [[1.4248367292861137, 0.0, -0.0005557552421618], [0.0, 1.426677521485123, 0.0188615443189732], [0.0, 0.0, -1.0]]
  eye_to_head [[0.9942073760240819, 0.0, -0.10747880469799379], [0.010021268493629042, 0.9956437105068295, 0.09269938460405774], [0.1070105959103494, -0.09323948588550035, 0.9898763208778758]]
  grow 0.6 cutoff 1
  distortion DISTORT_POLY3 center 0.00056 0.01886 coeffs [0.20536, 0.14656, 0.04697]
  distortion_red DISTORT_POLY3 center 0.00056 0.01886 coeffs [0.1823, 0.18034, 0.02817]
  distortion_blue DISTORT_POLY3 center 0.00056 0.01886 coeffs [0.24687, 0.08574, 0.08082]
ipd {'default_mm': 69}
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'scipy'


python -m pip install scipy 2>&1 | Select-Object -Last 1

Install scipy


[notice] To update, run: python.exe -m pip install --upgrade pip


Write

"""Spike S3 empirical analysis: discover Valve's actual POLY3 evaluation.

The 4 naive hypotheses all failed because the source-UV space is the
GetProjectionRaw frustum, not a grow-rescaled NDC. This script fits the
general radial model per eye/channel:

    t   = (2u-1, 2v-1)            panel NDC (v down, OpenVR viewport)
    r2  = |t - c|^2
    tan = a + s * (t - c) * d(r2)   (s, a 2-vector; tan = raw-projection
                                     tangent space, y-down: tan = T+v*(B-T))

with d either  poly: 1 + k1 r2 + k2 r2^2 + k3 r2^3
        or    recip: 1 / (1 + k1 r2 + k2 r2^2 + k3 r2^3)

Free params per channel: c(2) a(2) s(2) k(3) = 9. Ground truth: 4225 pts.
Then compares fitted c,k against the config block and fitted a,s against
intrinsics/projection_raw to recover the closed form.

Usage: python analyze.py distortion_dump_*.json config_*.json
"""

import json
import sys

import numpy as np
from scipy.optimize import least_squares

CHANNELS = {"r": "distortion_red", "g": "distortion", "b": "distortion_blue"}

def load_eye_points(eye_dump):
    uv, tgt = [], {"r": [], "g": [], "b": []}
    for pt in eye_dump["points"]:
        if pt is None:
            continue
        uv.append(pt["uv"])
        for ch in ("r", "g", "b"):
            tgt[ch].append(pt[ch])
    uv = np.array(uv)
    L, R, T, B = eye_dump["projection_raw"]
    tans = {}
    for ch in ("r", "g", "b"):
        suv = np.array(tgt[ch])
        # source UV -> tangent space of the raw projection (v=0 top -> T)
        tans[ch] = np.stack([L + suv[:, 0] * (R - L), T + suv[:, 1] * (B - T)], axis=1)
    return uv, tans

def model(params, t, form):
    cx, cy, ax, ay, sx, sy, k1, k2, k3 = params
    d0 = t - np.array([cx, cy])
    r2 = (d0 ** 2).sum(axis=1)
    p = 1 + r2 * (k1 + r2 * (k2 + r2 * k3))
    d = p if form == "poly" else 1.0 / p
    return np.array([ax, ay]) + np.array([sx, sy]) * d0 * d[:, None]

def fit(uv, tan, form, k0):
    t = 2 * uv - 1
    x0 = np.array([0.0, 0.0, 0.0, 0.0, 0.7, 0.7] + list(k0))

    def resid(params):
        return (model(params, t, form) - tan).ravel()

    return least_squares(resid, x0, method="lm", max_nfev=20000)

def main():
    dump = json.load(open(sys.argv[1]))
    cfg = json.load(open(sys.argv[2]))
    rt = dump["recommended_render_target"][0]

    for eye_idx, eye_dump in enumerate(dump["eyes"]):
        tte = cfg["tracking_to_eye_transform"][eye_idx]
        K = tte["intrinsics"]
        L, R, T, B = eye_dump["projection_raw"]
        print(f"\n=== {eye_dump['eye'].upper()} ===")
        print(f"projection_raw L{L:+.5f} R{R:+.5f} T{T:+.5f} B{B:+.5f}")
        print(f"intrinsics fx {K[0][0]:.5f} fy {K[1][1]:.5f} K02 {K[0][2]:+.5f} K12 {K[1][2]:+.5f}")
        print("config eye_to_head:")
        for row in tte["eye_to_head"]:
            print("   ", [round(x, 6) for x in row])
        print("dump eye_to_head:")
        for row in eye_dump["eye_to_head"]:
            print("   ", [round(x, 6) for x in row])

        uv, tans = load_eye_points(eye_dump)
        for ch in ("g", "r", "b"):
            blk = tte[CHANNELS[ch]]
            kcfg = blk["coeffs"]
            for form in ("poly", "recip"):
                res = fit(uv, tans[ch], form, kcfg)
                cx, cy, ax, ay, sx, sy, k1, k2, k3 = res.x
                # residual in source pixels
                err = res.fun.reshape(-1, 2)
                err_px = np.linalg.norm(err, axis=1) * rt / abs(R - L)
                print(f" {ch}/{form:5s} rms {err_px.std() + err_px.mean():7.3f}px max {err_px.max():8.3f}px | "
                      f"c ({cx:+.5f},{cy:+.5f}) a ({ax:+.5f},{ay:+.5f}) s ({sx:+.5f},{sy:+.5f})")
                print(f"          k_fit  [{k1:+.5f}, {k2:+.5f}, {k3:+.5f}]")
                print(f"          k_cfg  [{kcfg[0]:+.5f}, {kcfg[1]:+.5f}, {kcfg[2]:+.5f}]  "
                      f"center_cfg ({blk['center_x']:+.5f},{blk['center_y']:+.5f})  "
                      f"1/fx {1/K[0][0]:.5f} 1/fy {1/K[1][1]:.5f}")

if __name__ == "__main__":
    main()