# Phase 1: Extraction - Pattern Map **Mapped:** 2026-05-02 **Files analyzed:** ~40 new files (one-time scaffolding + reader/dnd/emit modules + tests + fixtures + output tree shape + repo-root config) **Analogs found:** 0 in-repo / total (this is the first phase to introduce code; all analogs are external) ## Brownfield Reality Check Before pattern excerpts: this repo has **no in-repo analogs whatsoever** for any Phase 1 file. There is: - No `package.json` anywhere (verified — `.planning/codebase/TESTING.md` lines 28-29). - No TypeScript file anywhere. - No test framework, test directory, CI config, linter config, or formatter config (verified — `.planning/codebase/CONVENTIONS.md` lines 9-13 + `TESTING.md` lines 13-30). - The only "code" in the tree is two Node mirror scripts (`legacy/.mirror.js`, `legacy/.mirror-fix.js`) — one-shot ad-hoc archival scripts, not a pattern source. Every Phase 1 file is therefore "**first of its kind**." The planner should treat the external references in §External Pattern Sources below as the authoritative analogs and copy semantics from them — particularly `org.lateralgm.file.GmFileReader` (Java) for the parser core. The `decomp/wiki/` markdown pages are **specification**, not a pattern source — they describe the wire format, not implementation idioms. The discipline-setting moment: Phase 1's conventions (strict TS, ISO-8859-1 string decode, sorted-keys JSON, numeric-prefix output paths, exact-pinned deps) become the project's defaults from Phase 4 onward. Get them right here. ## File Classification Files grouped by subsystem. "Analog" column links to the closest external reference (no in-repo match exists). All paths are relative to repo root. ### Subsystem A — `tools/extract-gmd/` source (TS) | New File | Role | Data Flow | Closest Analog | Match Quality | |----------|------|-----------|----------------|---------------| | `tools/extract-gmd/cli.ts` | CLI entrypoint | request-response (argv → dispatch) | none in-repo; std Node CLI shape | first of its kind | | `tools/extract-gmd/src/reader/BinaryReader.ts` | utility (binary primitive reader) | transform (Buffer + cursor) | LateralGM `org.lateralgm.file.GmStreamDecoder` | first of its kind (port semantics) | | `tools/extract-gmd/src/reader/header.ts` | parser module | transform (bytes → version) | `GmFileReader.readProjectFile` (header section) | first of its kind | | `tools/extract-gmd/src/reader/readProjectFile.ts` | orchestrator | sequential pipeline | `GmFileReader.readProjectFile` (full method) | first of its kind | | `tools/extract-gmd/src/reader/settings.ts` | parser module | transform | `GmFileReader.readSettings` | first of its kind | | `tools/extract-gmd/src/reader/sounds.ts` | parser module | transform | `GmFileReader.readSounds` | first of its kind | | `tools/extract-gmd/src/reader/sprites.ts` | parser module | transform (+ ZLIB inflate) | `GmFileReader.readSprites` | first of its kind | | `tools/extract-gmd/src/reader/backgrounds.ts` | parser module | transform (+ ZLIB inflate) | `GmFileReader.readBackgrounds` | first of its kind | | `tools/extract-gmd/src/reader/paths.ts` | parser module | transform | `GmFileReader.readPaths` | first of its kind | | `tools/extract-gmd/src/reader/scripts.ts` | parser module | transform (GML plaintext) | `GmFileReader.readScripts` | first of its kind | | `tools/extract-gmd/src/reader/datafiles.ts` | parser module | transform | `GmFileReader.readDataFiles` (v500+ guard) | first of its kind | | `tools/extract-gmd/src/reader/fonts.ts` | parser module | transform | `GmFileReader.readFonts` | first of its kind | | `tools/extract-gmd/src/reader/timelines.ts` | parser module | transform (calls `dnd/`) | `GmFileReader.readTimelines` | first of its kind | | `tools/extract-gmd/src/reader/objects.ts` | parser module | transform (calls `dnd/`) | `GmFileReader.readObjects` | first of its kind | | `tools/extract-gmd/src/reader/rooms.ts` | parser module | transform | `GmFileReader.readRooms` | first of its kind | | `tools/extract-gmd/src/dnd/readAction.ts` | parser module | transform (per-node binary) | `GmFileReader.readActions` + `decomp/wiki/04` | first of its kind | | `tools/extract-gmd/src/dnd/actionLookup.ts` | service (lookup) | request-response (ID → descriptor) | LateralGM `library/default/*.lib` XML loaders | first of its kind | | `tools/extract-gmd/src/dnd/transcompile.ts` | utility | transform (action → GML string) | LateralGM `LibAction.execInfo` mapping | first of its kind | | `tools/extract-gmd/src/emit/tree.ts` | utility (path layout) | request-response | none — D-07 dictates the shape | first of its kind | | `tools/extract-gmd/src/emit/json.ts` | utility (sorted-keys serializer) | transform | std Node `JSON.stringify` + recursive key sort | first of its kind | | `tools/extract-gmd/src/emit/png.ts` | utility (deterministic PNG) | transform (RGBA → PNG bytes) | sharp docs (pinned options) | first of its kind | | `tools/extract-gmd/src/emit/manifest.ts` | utility (SHA256 walker) | transform | POSIX `sha256sum` line format | first of its kind | | `tools/extract-gmd/src/emit/unknown-actions.ts` | utility (markdown index writer) | event-driven (collect-then-write) | none — D-11 dictates the shape | first of its kind | | `tools/extract-gmd/src/verify.ts` | CLI subcommand | transform + diff | `sha256sum -c` semantics | first of its kind | ### Subsystem B — `tools/extract-gmd/data/` (lookup tables) | New File | Role | Data Flow | Closest Analog | Match Quality | |----------|------|-----------|----------------|---------------| | `tools/extract-gmd/data/action-ids.json` | data (DnD lookup) | static read | LateralGM `library/default/*.lib` XML files | port-and-transform (D-03) | | `tools/extract-gmd/data/lateralgm-libs/*.lib` | data (vendored XML, optional staging) | static read | LateralGM upstream | direct copy with attribution | ### Subsystem C — `tools/extract-gmd/tests/` (vitest fixtures + specs) | New File | Role | Data Flow | Closest Analog | Match Quality | |----------|------|-----------|----------------|---------------| | `tools/extract-gmd/vitest.config.ts` | config | static | vitest docs (NodeNext, single-thread for fs determinism) | first of its kind | | `tools/extract-gmd/tests/fixtures/build-fixtures.ts` | fixture builder | transform (TS → tiny `.gmd`) | none — invented per RESEARCH.md Wave 0 | first of its kind | | `tools/extract-gmd/tests/fixtures/tiny-empty.gmd` | binary fixture | static | none — built programmatically | first of its kind | | `tools/extract-gmd/tests/fixtures/tiny-script.gmd` | binary fixture | static | none — built programmatically | first of its kind | | `tools/extract-gmd/tests/fixtures/tiny-sprite.gmd` | binary fixture | static | none — built programmatically | first of its kind | | `tools/extract-gmd/tests/reader/*.test.ts` | unit test (per block) | request-response | vitest + golden fixture pattern | first of its kind | | `tools/extract-gmd/tests/dnd/*.test.ts` | unit test (DnD) | request-response | vitest + descriptor compare | first of its kind | | `tools/extract-gmd/tests/emit/*.test.ts` | unit test (emit layer) | request-response | vitest + byte-identity compare | first of its kind | | `tools/extract-gmd/tests/golden//...` | snapshot tree | static | vitest snapshot dir convention | first of its kind | | `tools/extract-gmd/tests/integration/extract-client.test.ts` | integration test | end-to-end | none — full real `.gmd` run | first of its kind | | `tools/extract-gmd/tests/integration/extract-server.test.ts` | integration test | end-to-end | mirror of extract-client | first of its kind | | `tools/extract-gmd/tests/integration/reproducibility.test.ts` | integration test (EXT-07) | extract twice + diff hashes | none — invented per D-15/D-16 | first of its kind | | `tools/extract-gmd/tests/integration/tree-shape.test.ts` | structural lint | static | none — asserts shape per D-07 | first of its kind | ### Subsystem D — `tools/extract-gmd/` config | New File | Role | Data Flow | Closest Analog | Match Quality | |----------|------|-----------|----------------|---------------| | `tools/extract-gmd/package.json` | config (deps + scripts) | static | npm docs (`exact` versions, no `^`/`~`) | first of its kind | | `tools/extract-gmd/tsconfig.json` | config (strict TS, NodeNext) | static | TypeScript docs (strict + NodeNext + ES2022) | first of its kind | | `tools/extract-gmd/README.md` | doc | static | thin pointer to `decomp/TOOLS.md` | first of its kind | ### Subsystem E — `extracted//...` output tree (committed artifacts, generated) The shape is dictated by **D-07**. Files are not hand-written — the emit layer produces them deterministically. Per D-15 each path has a numeric `-` prefix preserving native source order. **Pattern: one file per resource; no monolithic blobs.** | Output Path Pattern | Role | Data Flow | Source-of-Pattern | |---------------------|------|-----------|-------------------| | `extracted//settings.json` | metadata | written once | D-07 + `emit/json.ts` | | `extracted//scripts/-.gml` | text artifact | one per script | D-07 + GML plaintext from block 7 | | `extracted//objects/-/meta.json` | metadata | per object | D-07 §objects | | `extracted//objects/-/events/.dnd.json` | DnD descriptor (canonical) | per event | D-09 (lossless) | | `extracted//objects/-/events/.gml` | transcompiled GML | per event | D-09, D-10 (read-primary) | | `extracted//sprites/-/meta.json` | metadata | per sprite | D-07 §sprites | | `extracted//sprites/-/frames/img_NN.png` | binary asset (deterministic PNG) | per subimage | D-12 + `emit/png.ts` | | `extracted//backgrounds/-/{meta.json,image.png}` | binary asset | per background | D-07 + D-12 | | `extracted//rooms/-/{meta,instances,tiles,backgrounds}.json + creation-code.gml` | composite | per room | D-07 §rooms | | `extracted//sounds/-/{meta.json, audio.}` | metadata + raw bytes | per sound | D-13 (embedded only; relative-path sounds skip audio bytes) | | `extracted//fonts/-/{meta.json, glyphs.png}` | metadata + raster strip | per font | D-14 | | `extracted//paths/-.json` | composite (one file) | per path | D-07 §paths (whole resource one file) | | `extracted//timelines/-/moments/.{dnd.json,gml}` | DnD descriptor + GML | per moment | D-07, D-09 | | `extracted//datafiles/-.` | raw bytes | per datafile | D-07 §datafiles (5.x+ feature, version-guarded) | | `extracted//UNKNOWN-ACTIONS.md` | markdown index | written iff any unknown ID | D-11 | | `extracted//MANIFEST.sha256` | reproducibility checkpoint | committed | D-15, D-16 | ### Subsystem F — repo-root config (modified, not new) | Modified File | Role | Data Flow | Existing Pattern | Change | |---------------|------|-----------|------------------|--------| | `.gitignore` | config | static | currently excludes `legacy/` (per CONVENTIONS.md line 13) | **extend**: add `.extracted-cache/`, `tools/*/node_modules/`, `tools/*/dist/` | | `package.json` (repo root, NEW) | config (workspace scripts) | static | does not exist yet | **create**: minimal repo-root with `extract:all` script invoking `tsx tools/extract-gmd/cli.ts` for both `.gmd` files | ### Subsystem G — `decomp/TOOLS.md` (documentation deliverable, EXT-08) | New File | Role | Data Flow | Closest Analog | Match Quality | |----------|------|-----------|----------------|---------------| | `decomp/TOOLS.md` | doc (thin wrapper, EXT-08) | static | `decomp/wiki/15-extraction-pipeline.md` (the substantive doc — TOOLS.md links + adds installation/provenance, not duplication) | in-repo analog: `decomp/wiki/*.md` markdown style (see §Pattern: Wiki Markdown) | ## Pattern Assignments Because no in-repo TS analogs exist, each pattern below is sourced from RESEARCH.md (which itself ports from `LateralGM/GmFileReader.java`) or from a project decision (D-NN). Excerpts are reproduced verbatim from RESEARCH.md so the planner can paste them directly into plan actions. --- ### `tools/extract-gmd/src/reader/BinaryReader.ts` (utility, transform) **Analog:** `org.lateralgm.file.GmStreamDecoder` (LateralGM upstream, Java). No in-repo analog. **Required surface** (mirror semantics, not method names — port to TS idioms): | TS method | Java mirror | Notes | |-----------|-------------|-------| | `readBool()` | `readBool()` | 1 byte, 0/1 | | `readByte()` | `read1()` | 1 byte unsigned | | `readInt32LE()` | `read4()` | signed/unsigned distinction matters; default signed, expose `readUint32LE` if any field needs it | | `readDouble()` | `readD()` | 8-byte IEEE 754 little-endian | | `readStr()` | `readStr()` | **4-byte length prefix**, decoded as **ISO-8859-1** (Node `Buffer.toString('latin1')`). NEVER UTF-8. (RESEARCH.md Pitfall 3, line 482-487) | | `readStr1()` | `readStr1()` | 1-byte length prefix variant — needed for some DnD argument values (RESEARCH.md A2, line 633-635) | | `readBytes(n)` | `read(n)` | Bound `n` to `(bufLen − cursor)` — defense vs. malicious length-prefix OOM (RESEARCH.md Security V5, line 758) | | `decompress(n)` | `decompress(read4())` | Read 4-byte length, slice, `zlib.inflateSync` — matches LateralGM length-prefixed inflate sub-stream framing | | `skip(n)` | n/a (Java uses `skipBytes`) | advance cursor without read; used for v800+ "last changed" timestamps | | `atEnd()` | n/a | cursor === bufLen — for defensive trailing-block parse | **Hard rule** (copy verbatim into the file as a comment): ```typescript // Source: org.lateralgm.file.GmStreamDecoder (LateralGM 1.8.234) // String decoding is hard-pinned to ISO-8859-1 (Node 'latin1'). See // RESEARCH.md Pitfall 3. Never use 'utf8' for resource names. ``` --- ### `tools/extract-gmd/src/reader/readProjectFile.ts` (orchestrator) **Analog:** `GmFileReader.readProjectFile` (LateralGM, Java). **Core pattern** (RESEARCH.md §Pattern 1, lines 280-309): ```typescript // src/reader/readProjectFile.ts // Source: github.com/IsmAvatar/LateralGM/blob/master/org/lateralgm/file/GmFileReader.java // (readProjectFile method, version dispatch) export function readProjectFile(buf: Buffer): ProjectFile { const r = new BinaryReader(buf); const identifier = r.readInt32LE(); if (identifier !== 1234321) throw new Error('Not a .gmd file'); const ver = r.readInt32LE(); if (![530, 542, 600, 701, 800, 810].includes(ver)) { throw new Error(`Unsupported version: ${ver}`); } // For 5.3a we expect ver === 530 (and possibly 542 — verify on real input). const project: ProjectFile = { version: ver }; project.settings = readSettings(r, ver); // block 2 project.sounds = readSounds(r, ver); // block 3 project.sprites = readSprites(r, ver); // block 4 (ZLIB) project.backgrounds = readBackgrounds(r, ver); // block 5 (ZLIB) project.paths = readPaths(r, ver); // block 6 project.scripts = readScripts(r, ver); // block 7 (GML) project.datafiles = readDataFiles(r, ver); // block 8 (v500+) project.fonts = readFonts(r, ver); // block 9 project.timelines = readTimelines(r, ver); // block 10 project.objects = readObjects(r, ver); // block 11 (DnD lives here) project.rooms = readRooms(r, ver); // block 12 if (!r.atEnd()) project.gameInfo = readGameInfo(r, ver); return project; } ``` **Anti-pattern** (RESEARCH.md line 425-426): never hard-code "v530 layout". Every block reader takes `ver` and dispatches per-version. Format version persists across IDE upgrades, so even our 5.3a `.gmd` may report 542+ if ever re-saved. --- ### `tools/extract-gmd/src/reader/sprites.ts` (parser, transform + ZLIB) **Analog:** `GmFileReader.readSprites` (LateralGM, Java). **Per-resource "exists" flag pattern** (RESEARCH.md §Pattern 2, lines 318-335) — applies to **every** resource block (sounds, sprites, backgrounds, paths, scripts, fonts, timelines, objects, rooms): ```typescript // src/reader/sprites.ts (canonical pattern; same shape for sounds/bgs/paths/etc.) // Source: GmFileReader.readSprites — github.com/IsmAvatar/LateralGM export function readSprites(r: BinaryReader, ver: number): Sprite[] { const count = r.readInt32LE(); const sprites: Sprite[] = []; for (let id = 0; id < count; id++) { const exists = r.readBool(); if (!exists) continue; // slot is empty but ID is consumed const name = r.readStr(); // ISO-8859-1, length-prefixed if (ver >= 800) r.skip(8); // "last changed" timestamp — ignore for determinism const sver = r.readInt32LE(); // sprite version // ... bbox, origin, mask, subimages (ZLIB pixel buffer per subimage) sprites.push({ id, name, /* ... */ }); } return sprites; } ``` **ZLIB pixel buffer pattern** (RESEARCH.md §Code Examples lines 535-562): ```typescript import { inflateSync } from 'node:zlib'; const subImageCount = r.readInt32LE(); const frames: Buffer[] = []; for (let f = 0; f < subImageCount; f++) { const compressedLen = r.readInt32LE(); const compressed = r.readBytes(compressedLen); const raw = inflateSync(compressed); // RGBA or palette per ver frames.push(raw); } ``` **CRITICAL ID-stability rule** (RESEARCH.md line 337-340): > Output filenames must preserve numeric IDs as zero-padded prefixes (e.g. `sprites/0017-sprPlayer/`) so: > 1. Filesystem listing matches native source order (D-15). > 2. Rooms cross-referencing object ID 17 still resolve unambiguously when sorted alphabetically. > 3. Round-tripability is preserved without needing a separate ID-mapping file. --- ### `tools/extract-gmd/src/dnd/readAction.ts` (parser, transform) **Analog:** `GmFileReader.readActions` (LateralGM) + `decomp/wiki/04-dnd-serialization.md`. **Core pattern** (RESEARCH.md §Pattern 3, lines 352-373): ```typescript // src/dnd/readAction.ts // Source: decomp/wiki/04-dnd-serialization.md export function readAction(r: BinaryReader): DnDAction { const actionId = r.readInt32LE(); const appliesTo = r.readInt32LE(); // -1 self, -2 other, >=0 object ID const isRelative = r.readBool(); const argCount = r.readInt32LE(); const argTypes = Array.from({ length: argCount }, () => r.readInt32LE()); const argValues = argTypes.map(t => readArgValue(r, t)); return { actionId, appliesTo, isRelative, argTypes, argValues }; } function readArgValue(r: BinaryReader, type: ArgType) { switch (type) { case ArgType.String: return r.readStr(); // null-terminated OR length-prefixed case ArgType.Real: return r.readDouble(); // 8-byte IEEE 754 LE case ArgType.Resource: return r.readInt32LE(); // resource ID case ArgType.Bool: return r.readBool(); default: throw new Error(`Unknown arg type: ${type}`); } } ``` **Open question to resolve in Wave 0** (RESEARCH.md A2, line 633-635 + line 375): string-arg encoding ambiguity. Hand-decode one short BNO event against LateralGM's GUI rendering before locking the choice. --- ### `tools/extract-gmd/src/emit/json.ts` (utility, transform) **Analog:** none. Invented per D-15. **Core pattern** (RESEARCH.md §Code Examples lines 565-582): ```typescript // src/emit/json.ts import { writeFileSync } from 'node:fs'; export function writeJsonDeterministic(path: string, value: unknown): void { const sorted = sortKeysRecursive(value); const json = JSON.stringify(sorted, null, 2) + '\n'; writeFileSync(path, json, 'utf8'); } function sortKeysRecursive(v: unknown): unknown { if (Array.isArray(v)) return v.map(sortKeysRecursive); if (v !== null && typeof v === 'object') { const o = v as Record; return Object.fromEntries(Object.keys(o).sort().map(k => [k, sortKeysRecursive(o[k])])); } return v; } ``` **Determinism contract** (D-15): sorted keys + 2-space indent + LF + single trailing newline + no `Date.now()` + no absolute paths. Test with a unit case that shuffles input key order and asserts identical output. --- ### `tools/extract-gmd/src/emit/png.ts` (utility, transform) **Analog:** sharp library docs. No in-repo analog. **Core pattern** (RESEARCH.md §Pattern 4, lines 404-417): ```typescript // src/emit/png.ts import sharp from 'sharp'; export async function writeDeterministicPng(path: string, w: number, h: number, rgba: Buffer): Promise { await sharp(rgba, { raw: { width: w, height: h, channels: 4 } }) .png({ compressionLevel: 9, palette: false, effort: 10, progressive: false, adaptiveFiltering: false, // Do NOT pass `quality` — let libvips compute filter strategy from compressionLevel only. }) .toFile(path); } ``` **Pin** sharp to exact version `0.34.5` in `package.json` (no `^` or `~`). Surface BOTH `sharp` package version AND `sharp.versions.vips` (libvips revision) in the MANIFEST.sha256 header — RESEARCH.md line 137-138, 420. **Cross-OS drift mitigation** (RESEARCH.md Pitfall 4, lines 489-499): - Designate Linux x64 as canonical platform. - CI pass-fail only on Linux; mac/Windows treat hash drift as soft warning. - Document this rule in `decomp/TOOLS.md`. --- ### `tools/extract-gmd/src/emit/manifest.ts` (utility, transform) **Analog:** POSIX `sha256sum` line format. **Core pattern** (RESEARCH.md §Code Examples lines 587-613): ```typescript // src/emit/manifest.ts import { readdirSync, readFileSync, statSync, writeFileSync } from 'node:fs'; import { createHash } from 'node:crypto'; import { join, relative, posix } from 'node:path'; export function writeManifest(treeDir: string, sharpVersion: string, libvipsVersion: string): void { const lines: string[] = []; lines.push(`# extract-gmd MANIFEST`); lines.push(`# sharp=${sharpVersion}, libvips=${libvipsVersion}`); for (const f of walkSorted(treeDir)) { const rel = posix.normalize(relative(treeDir, f).replace(/\\/g, '/')); if (rel === 'MANIFEST.sha256') continue; const hash = createHash('sha256').update(readFileSync(f)).digest('hex'); lines.push(`${hash} ${rel}`); } writeFileSync(join(treeDir, 'MANIFEST.sha256'), lines.join('\n') + '\n', 'utf8'); } function* walkSorted(dir: string): Generator { const entries = readdirSync(dir).sort(); // POSIX-collated lexicographic for (const name of entries) { const full = join(dir, name); if (statSync(full).isDirectory()) yield* walkSorted(full); else yield full; } } ``` **Note:** Manifest *walks* the tree in alphabetical order (so the manifest itself is deterministic), but the *content* of each directory was *written* in native-source-order (numeric-prefix preserves it). Two distinct ordering rules — do not conflate. --- ### `tools/extract-gmd/cli.ts` (CLI entrypoint, request-response) **Analog:** none. Standard Node CLI shape. **Required commands** (D-18): - `extract ` — parse + write tree + write MANIFEST.sha256 - `verify ` — recompute SHA256 + diff vs committed `MANIFEST.sha256`; non-zero exit on drift **Pattern skeleton** (no upstream excerpt — invented): ```typescript // tools/extract-gmd/cli.ts import { readFileSync } from 'node:fs'; import { readProjectFile } from './src/reader/readProjectFile.js'; import { emitTree } from './src/emit/tree.js'; import { verify } from './src/verify.js'; async function main(argv: string[]) { const [, , cmd, ...rest] = argv; switch (cmd) { case 'extract': { const [input, outDir] = rest; const buf = readFileSync(input); const proj = readProjectFile(buf); await emitTree(outDir, proj); return; } case 'verify': { const [srcDir] = rest; const ok = await verify(srcDir); process.exit(ok ? 0 : 1); } default: console.error('Usage: extract-gmd extract '); console.error(' extract-gmd verify '); process.exit(2); } } main(process.argv).catch(e => { console.error(e); process.exit(1); }); ``` --- ### `tools/extract-gmd/package.json` (config) **Analog:** none. Standard npm shape. **Required pins** (RESEARCH.md §Standard Stack lines 92-101 + §Installation lines 124-131; RESEARCH.md line 132-133 "Pin exact versions (no `^` / `~`)"): ```json { "name": "extract-gmd", "version": "0.1.0", "private": true, "type": "module", "bin": { "extract-gmd": "./cli.ts" }, "scripts": { "extract": "tsx cli.ts extract", "verify": "tsx cli.ts verify", "test": "vitest run --reporter=basic", "test:full": "vitest run" }, "dependencies": { "sharp": "0.34.5" }, "devDependencies": { "typescript": "5.6.0", "tsx": "4.21.0", "vitest": "4.1.5", "@types/node": "25.6.0" } } ``` **No `^`, no `~`, no `*`, no ranges anywhere.** EXT-07 byte-determinism depends on it. --- ### `tools/extract-gmd/tsconfig.json` (config) **Analog:** TypeScript handbook. ```json { "compilerOptions": { "target": "ES2022", "module": "NodeNext", "moduleResolution": "NodeNext", "strict": true, "esModuleInterop": true, "skipLibCheck": true, "noUncheckedIndexedAccess": true, "exactOptionalPropertyTypes": true, "forceConsistentCasingInFileNames": true, "resolveJsonModule": true }, "include": ["cli.ts", "src/**/*.ts", "tests/**/*.ts"] } ``` `noUncheckedIndexedAccess` matters for the binary parser: array reads return `T | undefined`, surfacing off-by-one bugs at compile time instead of in production output bytes. --- ### `decomp/TOOLS.md` (doc, EXT-08) **Analog:** in-repo `decomp/wiki/*.md` — same markdown style. **This is the only file in Phase 1 with an actual in-repo style analog.** **Style observations from `decomp/wiki/`** (CONVENTIONS.md lines 149-160): - ATX-style headers (`#`, `##`). - Tables for at-a-glance reference. - Inline code spans for filenames, paths, identifiers. - Backlinks via relative paths (`../GameMaker 5.3a Reverse Engineering Pipeline.md`). - "See also" footer section (per `decomp/wiki/README.md` convention). **Content scope** (D-19, RESEARCH.md line 28-29 + line 194): > `decomp/TOOLS.md` is a **thin wrapper** around `decomp/wiki/15-extraction-pipeline.md` — does NOT duplicate content. Adds: installation/provenance notes, WinXP VM as Rank-3 fallback procedure, `tools/extract-gmd` documented as the project's primary daily extractor. **Required sections** (RESEARCH.md line 711): - `Rank 1`, `Rank 2`, `Rank 3`, `Rank 4` headers (must be greppable for the lint script) - A `tools/extract-gmd` section explicitly naming the CLI as the primary daily extractor - Provenance note for LateralGM 1.8.234 + sharp 0.34.5 + libvips revision (matches MANIFEST header) --- ## Shared Patterns (apply across all relevant files) ### 1. ISO-8859-1 string decoding (everywhere strings are read) **Source:** RESEARCH.md Pitfall 3 lines 482-487 + the `readStr` excerpt above. **Apply to:** `BinaryReader.ts` (`readStr`, `readStr1`), every parser block, `dnd/readAction.ts` arg-value reader. **Rule:** `Buffer.toString('latin1')` ONLY. NEVER `'utf8'`. Add a unit test that round-trips byte `0xA9` → `"©"`. ### 2. Native-source-order preservation (every emit path) **Source:** D-15, RESEARCH.md Pitfall 5 lines 502-506. **Apply to:** `emit/tree.ts` and every caller. Every output dir/file gets a zero-padded numeric prefix `-`. **Never** sort resources alphabetically when emitting; the manifest walker is the only place lexicographic sort is allowed (and that's after the tree is laid down). ### 3. Version-dispatched parsing (every block reader) **Source:** RESEARCH.md Anti-Patterns line 425. **Apply to:** `reader/settings.ts`, `sounds.ts`, `sprites.ts`, `backgrounds.ts`, `paths.ts`, `scripts.ts`, `datafiles.ts`, `fonts.ts`, `timelines.ts`, `objects.ts`, `rooms.ts`. **Rule:** Every block reader signature is `(r: BinaryReader, ver: number) => …`. Dispatch on `ver` for v530 vs v542+ vs v600+ vs v800+ field differences (timestamps appear at v800, datafiles at v500+, etc. — verify each against `GmFileReader`). ### 4. Byte-cursor invariant (every block reader, optional but recommended) **Source:** RESEARCH.md Pitfall 2 lines 470-480. **Apply to:** every block reader. **Rule:** Where `GmFileReader` writes a known post-block offset, mirror it as `assert(r.cursor === expected)`. Cascading parse drift is the deepest correctness risk in Phase 1. ### 5. Bounded reads (defense vs. malicious `.gmd`) **Source:** RESEARCH.md Security V5 line 758. **Apply to:** `BinaryReader.readBytes` and `BinaryReader.decompress`. **Rule:** Bound `readBytes(n)` to `(bufLen − cursor)`; cap inflated size to 64 MB per buffer. Phase 1 reads only the two trusted source `.gmd` files in practice, but the parser will be re-used for snapshots later (Phase 3+) — bake defense in now. ### 6. Path-component sanitization (every emit caller) **Source:** RESEARCH.md Security STRIDE table line 768. **Apply to:** `emit/tree.ts`. **Rule:** Strip `/`, `\`, `..`, control chars from resource names before using them in path components. Reject names that resolve to empty or pure punctuation; substitute `_` and append `` prefix anyway. ### 7. Determinism reflex (everywhere bytes are written) **Source:** D-15 + RESEARCH.md §Pattern 4 line 378. **Apply to:** `emit/json.ts`, `emit/png.ts`, `emit/manifest.ts`, scripts/GML writer (when added). | Surface | Defense | |---------|---------| | JSON | Sorted keys, 2-space indent, LF, single trailing newline, no `Date.now()` | | GML | LF line endings, preserve original whitespace from `.gmd` plaintext | | PNG | Pinned sharp + surfaced libvips version, `compressionLevel:9, palette:false, effort:10, progressive:false, adaptiveFiltering:false` | | Filesystem | Native source order via `-` prefix; never sort by name on emit | | MANIFEST | Sort SHA256 lines by relative path (POSIX `/`, LF) for stable diff | ### 8. Unknown-action graceful degradation **Source:** D-11 + RESEARCH.md Pitfall 6 lines 508-513. **Apply to:** `dnd/transcompile.ts`, `emit/unknown-actions.ts`. **Rule:** Unknown Action_ID → emit only `.dnd.json` + stub `.gml` containing `// UNKNOWN ACTION_ID=; see .dnd.json` + append entry to `extracted//UNKNOWN-ACTIONS.md`. **Never crash.** A non-empty UNKNOWN-ACTIONS.md is expected output and a Phase-2 forcing function. ### 9. Magic-number validation up front **Source:** RESEARCH.md Security V5 + line 478. **Apply to:** `cli.ts`, `reader/header.ts`. **Rule:** First 4 bytes must equal `1234321` (0x12D687 LE). Reject `.gmd`-extension-but-bad-magic with a clear error before any further parsing. ### 10. Wiki markdown style (only `decomp/TOOLS.md`) **Source:** in-repo — `decomp/wiki/*.md` (CONVENTIONS.md §Subsystem 6). **Apply to:** `decomp/TOOLS.md`. **Rule:** ATX headers, tables, relative-path backlinks, "See also" footer, inline code spans for filenames. ## External Pattern Sources Listed because this phase has no in-repo TS/test analogs. Planner should treat these as the primary references. | Source | Used For | Where to Find | |--------|----------|---------------| | `org.lateralgm.file.GmFileReader` (Java) | Block walker semantics — every `reader/*.ts` ports a method from here | `github.com/IsmAvatar/LateralGM` master branch; tag `1.8.234` | | `org.lateralgm.file.GmStreamDecoder` (Java) | `BinaryReader.ts` primitives + ISO-8859-1 pin | same repo | | LateralGM `library/default/*.lib` (XML) | DnD `Action_ID` → engine-function mapping for `data/action-ids.json` (D-03) | `org/lateralgm/resources/library/default/` in same repo | | GmkSplitter | Output tree shape per D-07 | `github.com/Medo42/Gmk-Splitter` (pattern reference only — not vendored) | | `decomp/wiki/03-gmd-format.md` | Top-level block layout, sequential parsing rule, ZLIB blocks, Int32 LE, IEEE 754 doubles | in-repo | | `decomp/wiki/04-dnd-serialization.md` | Per-node DnD binary layout | in-repo | | `decomp/wiki/11-tool-lateralgm.md` | Names `GmFileReader` as canonical reference parser | in-repo | | `decomp/wiki/12-tool-gmksplitter.md` | VCS-tree pattern (the "shape" half of D-07) | in-repo | | `decomp/wiki/13-modern-tool-incompat.md` | **HARD BAN list:** UTMT, Altar.NET, any FORM-chunk Studio tool | in-repo | | `decomp/wiki/14-gb1-backups.md` | `.gb1` is byte-identical to `.gmd` — parser should accept either extension | in-repo | | `decomp/wiki/15-extraction-pipeline.md` | Rank 1-4 procedure — informs `decomp/TOOLS.md` content but is NOT duplicated by it | in-repo | | sharp library docs | Deterministic PNG options | npmjs.com/package/sharp | | vitest docs | Snapshot testing, golden fixtures, single-thread fs determinism | vitest.dev | ## No Analog Found (every Phase 1 file) This is the entire file list. Per the brownfield reality check at the top, **no in-repo TS/test analog exists for any file in this phase except `decomp/TOOLS.md`** (whose markdown style mirrors `decomp/wiki/*.md`). Planner should rely on: 1. The verbatim code excerpts in §Pattern Assignments above (sourced from RESEARCH.md, themselves ported from LateralGM upstream). 2. The shared rules in §Shared Patterns. 3. The external references in §External Pattern Sources. This is appropriate for the first phase of a brownfield-archive project introducing a brand-new toolchain. Phase 4+ files will have rich in-repo analogs because Phase 1 is establishing them. ## Metadata **Analog search scope:** entire repo tree (`decomp/`, `legacy/`, `.planning/`, root). Confirmed via `.planning/codebase/STRUCTURE.md`, `CONVENTIONS.md`, `TESTING.md`. **Files scanned for analogs:** all `*.ts`, `*.js`, `*.json`, `*.md`, `*.cs`, `*.gml` in tree. **In-repo TS/test analog count:** 0 (zero) — confirmed by TESTING.md lines 13-30 (no `package.json`, no test framework, no CI) and CONVENTIONS.md lines 9-13 (no editorconfig, eslint, prettier, gitattributes anywhere). **Pattern extraction date:** 2026-05-02