# Phase 11: Frame Repository and Subset Grid Compilation - Research

**Researched:** 2026-04-13
**Domain:** Disk-backed frame storage, subset querying, on-demand grid compilation
**Confidence:** HIGH

## Summary

Phase 11 decouples capture from grid compilation by introducing a disk-backed frame repository. Instead of holding frames in RAM (current `CaptureSession.frames: CaptureFrame[]`), repository captures write JPEG files to disk as they're captured. Agents then query, filter, and compile subsets of stored frames into grids on demand.

The implementation is architecturally straightforward because it builds on well-established patterns already in the codebase: the `ProfileManager` pattern for file I/O with JSON manifests, the `SessionManager` pattern for lifecycle management, the `Scheduler` for timed capture, and `compileGrid()` for grid compilation. No new external dependencies are needed -- Node.js `fs/promises` handles all disk operations, and the existing `sharp`, `pixel-compare`, and `idle-compressor` modules provide all processing capabilities.

The main complexity lies in (1) the five query modes with even-distribution `max_frames` sampling, (2) concurrent access to manifests during active capture, and (3) retention-based cleanup without races. These are solved with straightforward algorithms and the existing `persistQueue` serialization pattern from `ProfileManager`.

**Primary recommendation:** Implement as a new `src/repository/` module directory containing `RepositoryManager` (session CRUD + manifest persistence), `RepositoryScheduler` (disk-writing capture loop), `frame-query.ts` (subset selection + sampling), and register 5 new MCP tools in `server.ts`.

<user_constraints>
## User Constraints (from CONTEXT.md)

### Locked Decisions
- **D-01:** Five query modes for frame subset selection (time range, time+length, index range, index+count, all)
- **D-02:** Optional `max_frames` ceiling with even distribution (uniform sampling)
- **D-03:** Query mode inferred from parameters; ambiguous combos rejected
- **D-04:** Frames stored as individual JPEG files in `.screen-timelapse/frames/{session-id}/{index}.jpg`
- **D-05:** Manifest JSON per session with session ID, config, start timestamp, frame manifest array, state, skipped frames
- **D-06:** Repository root configurable via `SCREEN_TIMELAPSE_FRAMES_PATH` env var, default `.screen-timelapse/frames/`
- **D-07:** 24-hour retention with auto-purge on server start and lazy cleanup on list/query
- **D-08:** Explicit delete tool available before 24h window
- **D-09:** Retention from session creation time, not last access
- **D-10:** `start_repository_capture` accepts same params as `start_capture` including profiles
- **D-11:** Higher default `max_frames` (200 vs 50) for repository captures
- **D-12:** Async capture -- returns session ID immediately, agents poll status
- **D-13:** Session states: capturing, complete, error; live tail supported for in-progress sessions
- **D-14:** `compile_subset_grid` returns base64 JPEG grid
- **D-15:** Reuses existing `compileGrid()` function from `grid-compiler.ts`
- **D-16:** Multiple subset grids from same session; frames not consumed
- **D-17:** Diagnostic options per-compilation, not per-session
- **D-18:** `list_repository_sessions` returns session metadata + retention deadline
- **D-19:** `list_repository_frames` returns frame manifest with optional change metric
- **D-20:** `list_repository_frames` accepts same query params as `compile_subset_grid`
- **D-21:** `compile_subset_grid` supports `label` parameter for title overlay
- **D-22:** `list_repository_frames` returns `change_summary` for session
- **D-23:** `start_repository_capture` supports `session_name` for named references
- **D-24:** `delete_repository_session` tool with disk cleanup
- **D-25:** `compile_subset_grid` returns metadata alongside grid

### Claude's Discretion
- Internal file naming conventions (zero-padded indices, etc.)
- Manifest JSON structure and field ordering
- Exact change detection algorithm for D-19
- Periodic cleanup interval
- Whether `get_capture_status` returns partial frame count for in-progress repository captures
- Error message wording
- Sort order for session and frame listings

### Deferred Ideas (OUT OF SCOPE)
- Cross-session frame comparison (mixing frames from different sessions)
- Frame annotation (agent-labeled individual frames)
- Streaming grid updates (live-updating grid resource)
</user_constraints>

## Standard Stack

### Core (Already in Project)
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| Node.js `fs/promises` | Built-in | File I/O for JPEG frames and manifest JSON | No external dependency needed; `readFile`, `writeFile`, `mkdir`, `readdir`, `rm`, `stat` cover all operations [VERIFIED: Node.js built-in] |
| sharp | ^0.34.5 | Grid compilation, label overlay | Already used by `compileGrid()` [VERIFIED: package.json] |
| zod | ^3.25.0 | Tool input validation | Already used for all MCP tools [VERIFIED: package.json] |
| crypto | Built-in | UUID generation for session IDs | Already used in `SessionManager` [VERIFIED: session-manager.ts] |

### No New Dependencies Required
This phase requires zero new npm packages. All functionality is covered by existing dependencies and Node.js built-ins. [VERIFIED: codebase analysis]

## Architecture Patterns

### Recommended Project Structure
```
src/
├── repository/               # NEW: Frame repository module
│   ├── repository-manager.ts # Session CRUD, manifest persistence, retention cleanup
│   ├── repository-scheduler.ts # Disk-writing capture loop (adapts scheduler.ts)
│   ├── frame-query.ts        # Subset selection, even-distribution sampling
│   └── repository-types.ts   # Repository-specific types and Zod schemas
├── capture/                  # EXISTING: unchanged
├── processing/               # EXISTING: grid-compiler.ts reused directly
├── profiles/                 # EXISTING: unchanged
└── server.ts                 # MODIFIED: register 5 new tools
```

### Pattern 1: RepositoryManager (Singleton with File Persistence)
**What:** A module-level singleton managing repository sessions on disk, following `ProfileManager` pattern.
**When to use:** All repository session CRUD operations.
**Key design points:**
- Lazy-loads session manifests from disk on first access [VERIFIED: ProfileManager pattern in profile-manager.ts]
- Uses `persistQueue` pattern for serialized writes to prevent concurrent write races [VERIFIED: ProfileManager.persist() in profile-manager.ts]
- Session lookup by ID or by name (D-23) using a `Map<string, RepositorySession>` with a secondary `Map<string, string>` for name-to-ID lookup
- Retention cleanup triggered on `listSessions()` and `getSession()` calls (lazy purge per D-07)

```typescript
// Pattern from ProfileManager -- adapt for repository
export class RepositoryManager {
  private sessions: Map<string, RepositorySession> | null = null;
  private readonly basePath: string;
  private persistQueue: Promise<void> = Promise.resolve();

  constructor(basePath?: string) {
    this.basePath = path.resolve(
      basePath ??
        process.env.SCREEN_TIMELAPSE_FRAMES_PATH ??
        ".screen-timelapse/frames",
    );
  }
}
```

### Pattern 2: Disk-Writing Scheduler (Adapted from scheduler.ts)
**What:** A modified capture loop that writes JPEG buffers to disk instead of accumulating them in `session.frames[]`.
**When to use:** `start_repository_capture` tool.
**Key design points:**
- Reuses the self-correcting timer logic from `scheduler.ts` [VERIFIED: scheduler.ts lines 72-78]
- Instead of `session.frames.push({ buffer, elapsedMs, index })`, writes buffer to `{session-dir}/{index}.jpg` and appends metadata to manifest
- Manifest is persisted after each frame write so partial sessions are queryable (D-13 live tail)
- The `CaptureFrame.buffer` is the JPEG buffer from the capture pipeline -- write directly, no re-encoding needed [VERIFIED: CONTEXT.md code_context section]

### Pattern 3: Frame Query with Even Distribution
**What:** Subset selection across five query modes with uniform sampling when `max_frames` is specified.
**When to use:** `list_repository_frames` and `compile_subset_grid` tools.
**Algorithm for even distribution (D-02):**
```typescript
function evenDistribute(frames: FrameMetadata[], maxFrames: number): FrameMetadata[] {
  if (frames.length <= maxFrames) return frames;
  const result: FrameMetadata[] = [];
  const step = (frames.length - 1) / (maxFrames - 1);
  for (let i = 0; i < maxFrames; i++) {
    result.push(frames[Math.round(i * step)]);
  }
  return result;
}
```
This ensures first and last frames of the range are always included, with evenly-spaced frames between them.

### Pattern 4: Loading Disk Frames into CaptureFrame[]
**What:** Read JPEG files from disk and construct `CaptureFrame[]` for `compileGrid()` compatibility.
**When to use:** `compile_subset_grid` tool.
```typescript
// Load selected frames from disk into CaptureFrame[] format
async function loadFrames(
  sessionDir: string,
  frameEntries: FrameMetadata[],
): Promise<CaptureFrame[]> {
  return Promise.all(
    frameEntries.map(async (entry) => ({
      buffer: await readFile(path.join(sessionDir, entry.filename)),
      elapsedMs: entry.elapsed_ms,
      index: entry.index,
    })),
  );
}
```

### Anti-Patterns to Avoid
- **Loading all frames into memory at once:** For a 200-frame session at ~100KB/frame = 20MB. Still manageable, but load only the subset needed for the query, not the full session.
- **Re-encoding JPEG frames on disk write:** Frames are already JPEG-compressed in the capture pipeline. Write the buffer directly, no sharp processing needed during capture.
- **Updating manifest synchronously in tick loop:** The manifest write must be async and not block the next capture tick. Write manifest after `writeFile` for the frame completes, but don't await it in the critical timing path -- queue it.

## Don't Hand-Roll

| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| Grid compilation | Custom compositing | `compileGrid()` from `grid-compiler.ts` | Already handles resize, timestamp overlays, delta highlighting, idle compression [VERIFIED: grid-compiler.ts] |
| Pixel comparison | Custom diff algorithm | `compareFrames()` from `pixel-compare.ts` | Already handles RGBA extraction, threshold-based comparison, returns `changedFraction` [VERIFIED: pixel-compare.ts] |
| Idle detection | Custom idle analysis | `compressIdleFrames()` from `idle-compressor.ts` | Already detects idle stretches with configurable threshold [VERIFIED: idle-compressor.ts] |
| UUID generation | Custom ID scheme | `crypto.randomUUID()` | Already used in `SessionManager` [VERIFIED: session-manager.ts] |
| Profile resolution | Custom param merging | `resolveScreenshotProfile()` + `resolveTimingProfile()` | Already handles ?? override semantics [VERIFIED: profile-resolver.ts] |
| Timer drift correction | Custom timing | Adapt `scheduler.ts` pattern | Already handles self-correcting setTimeout [VERIFIED: scheduler.ts] |

**Key insight:** This phase is primarily a storage and querying layer on top of existing capture and processing infrastructure. The existing modules do all the heavy lifting.

## Common Pitfalls

### Pitfall 1: Manifest Write Races During Capture
**What goes wrong:** Multiple async manifest writes overlap, causing data corruption or lost frame entries.
**Why it happens:** Each captured frame triggers a manifest update. At fast intervals (100ms), writes can overlap.
**How to avoid:** Use the `persistQueue` serialization pattern from `ProfileManager` -- chain writes via `this.persistQueue = this.persistQueue.then(async () => { ... })`. [VERIFIED: profile-manager.ts lines 98-108]
**Warning signs:** Missing frame entries in manifest, manifest JSON parse errors.

### Pitfall 2: Blocking Capture Loop with Disk I/O
**What goes wrong:** `writeFile` for a frame takes longer than the capture interval, causing drift or missed frames.
**Why it happens:** JPEG files at 80% quality are typically 50-200KB. Disk writes are fast but not instant, especially on spinning disks.
**How to avoid:** Fire-and-forget the disk write (track with a promise for error handling, but don't `await` it before scheduling the next capture). The scheduler's drift-correction logic already handles timing jitter. [VERIFIED: scheduler.ts drift correction pattern]
**Warning signs:** Increasing drift warnings in logs, frames captured later than expected.

### Pitfall 3: Retention Cleanup Deleting Active Sessions
**What goes wrong:** A session created 23.5 hours ago has its frames purged while the agent is still using it.
**Why it happens:** Lazy cleanup on list/query checks creation time, not whether frames are being actively compiled.
**How to avoid:** Retention check is simple: `Date.now() - session.createdAt > 24 * 60 * 60 * 1000`. No need to track "last access" (D-09 explicitly states creation-time-based). The 24h window is generous enough. Log when sessions are purged so agents can diagnose "session not found" errors.
**Warning signs:** "Session not found" errors for recently-created sessions.

### Pitfall 4: Session Name Collisions
**What goes wrong:** Two sessions with the same name cause lookup ambiguity.
**Why it happens:** D-23 allows optional session names as alternative identifiers.
**How to avoid:** Session names must be unique across active sessions. Reject `start_repository_capture` if a session with the same name already exists. Session IDs remain the canonical identifier; names are convenience aliases.
**Warning signs:** Wrong session returned when querying by name.

### Pitfall 5: Query Parameter Validation Gaps
**What goes wrong:** Ambiguous parameter combinations produce unexpected results.
**Why it happens:** D-03 requires rejecting ambiguous combos (e.g., both `to_ms` and `length_ms`), but validation logic might miss edge cases.
**How to avoid:** Implement query mode detection as an explicit function that returns a discriminated union. Map each valid combination to exactly one mode. Return a validation error for any unrecognized combination.
**Warning signs:** Silent wrong behavior when agents mix query params.

### Pitfall 6: `compileGrid()` Input Format Mismatch
**What goes wrong:** Frames loaded from disk don't match `CaptureFrame` interface expectations.
**Why it happens:** `CaptureFrame.buffer` expects a Buffer (JPEG), `compileGrid()` processes through sharp which handles JPEG input fine. But the `index` field on disk-loaded frames might not be sequential after subset selection.
**How to avoid:** When loading frames for compilation, preserve original `index` and `elapsedMs` from the manifest. `compileGrid()` uses these for timestamp overlays and doesn't care about sequential indices. [VERIFIED: grid-compiler.ts uses `processedFrames[i].elapsedMs` for timestamps]
**Warning signs:** Wrong timestamps in compiled grids.

## Code Examples

### Manifest JSON Structure (D-05)
```typescript
// Source: Designed per CONTEXT.md D-04, D-05
interface RepositoryManifest {
  sessionId: string;
  sessionName?: string;          // D-23: optional friendly name
  captureConfig: CaptureConfig;  // reuse existing type
  startedAt: string;             // ISO 8601
  state: SessionState;           // "capturing" | "complete" | "error"
  error?: string;
  frameCount: number;
  frames: FrameEntry[];
  skippedFrames: SkippedFrame[];  // reuse existing type
}

interface FrameEntry {
  index: number;
  elapsed_ms: number;
  filename: string;              // e.g., "000042.jpg"
  size_bytes: number;
}
```

### Query Mode Detection (D-01, D-03)
```typescript
// Source: Designed per CONTEXT.md D-01, D-03
type QueryMode =
  | { mode: "time_range"; from_ms: number; to_ms: number }
  | { mode: "time_length"; from_ms: number; length_ms: number }
  | { mode: "index_range"; from_index: number; to_index: number }
  | { mode: "index_count"; from_index: number; frame_count: number }
  | { mode: "all" };

function detectQueryMode(params: QueryParams): QueryMode | { error: string } {
  const hasFromMs = params.from_ms !== undefined;
  const hasToMs = params.to_ms !== undefined;
  const hasLengthMs = params.length_ms !== undefined;
  const hasFromIndex = params.from_index !== undefined;
  const hasToIndex = params.to_index !== undefined;
  const hasFrameCount = params.frame_count !== undefined;

  // Reject ambiguous combos
  if (hasToMs && hasLengthMs) return { error: "Cannot specify both to_ms and length_ms" };
  if (hasToIndex && hasFrameCount) return { error: "Cannot specify both to_index and frame_count" };
  if ((hasFromMs || hasToMs || hasLengthMs) && (hasFromIndex || hasToIndex || hasFrameCount)) {
    return { error: "Cannot mix time-based and index-based query params" };
  }

  if (hasFromMs && hasToMs) return { mode: "time_range", from_ms: params.from_ms!, to_ms: params.to_ms! };
  if (hasFromMs && hasLengthMs) return { mode: "time_length", from_ms: params.from_ms!, length_ms: params.length_ms! };
  if (hasFromIndex && hasToIndex) return { mode: "index_range", from_index: params.from_index!, to_index: params.to_index! };
  if (hasFromIndex && hasFrameCount) return { mode: "index_count", from_index: params.from_index!, frame_count: params.frame_count! };

  // All frames (no query params)
  if (!hasFromMs && !hasToMs && !hasLengthMs && !hasFromIndex && !hasToIndex && !hasFrameCount) {
    return { mode: "all" };
  }

  return { error: "Incomplete query parameters" };
}
```

### Even Distribution Sampling (D-02)
```typescript
// Source: Designed per CONTEXT.md D-02
function sampleFrames<T>(frames: T[], maxFrames: number): T[] {
  if (frames.length <= maxFrames) return frames;
  if (maxFrames === 1) return [frames[0]];

  const result: T[] = [];
  const step = (frames.length - 1) / (maxFrames - 1);
  for (let i = 0; i < maxFrames; i++) {
    result.push(frames[Math.round(i * step)]);
  }
  return result;
}
```

### Label Overlay for compile_subset_grid (D-21)
```typescript
// Source: Extends existing timestamp-overlay.ts pattern
// Label is rendered as a title bar at the top of the compiled grid
async function addLabelOverlay(
  gridBuffer: Buffer,
  label: string,
  gridWidth: number,
): Promise<Buffer> {
  const { createTimestampOverlay } = await import("./timestamp-overlay.js");
  const overlay = await createTimestampOverlay(label, gridWidth, 30);
  // Composite at top-center of the grid
  return sharp(gridBuffer)
    .composite([{ input: overlay, left: 4, top: 4 }])
    .jpeg()
    .toBuffer();
}
```

### Change Summary Computation (D-22)
```typescript
// Source: Reuses pixel-compare.ts and idle-compressor.ts patterns
interface ChangeSummary {
  totalFrames: number;
  framesWithChanges: number;
  longestIdleMs: number;
  mostActiveStartMs: number;
  mostActiveEndMs: number;
}
```

## State of the Art

| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| In-memory frame storage | Disk-backed repository | Phase 11 | Removes RAM constraint, enables 200+ frame sessions |
| Capture + compile coupled | Decoupled capture/compile | Phase 11 | Multiple grid compilations from single capture |
| Single grid per session | On-demand subset grids | Phase 11 | Agents can explore different time windows |

## Assumptions Log

| # | Claim | Section | Risk if Wrong |
|---|-------|---------|---------------|
| A1 | JPEG frame files at quality 80 are typically 50-200KB each | Pitfall 2 | Disk usage estimates may be off; 200 frames at 200KB = 40MB max, still reasonable |
| A2 | Fire-and-forget disk writes won't cause data loss on normal shutdown | Pitfall 2 | Frames could be written but manifest not updated; recoverable by scanning directory |
| A3 | `sharp` handles JPEG input buffers the same as PNG for `compileGrid()` | Pattern 4 | Grid compilation could fail; but sharp auto-detects format so this is very low risk [VERIFIED: sharp docs say format auto-detected] |

## Open Questions

1. **Zero-padded index format**
   - What we know: Frame filenames need to sort lexicographically for directory listings
   - Recommendation: Use 6-digit zero-padding (`000042.jpg`) to support up to 999,999 frames. Overkill for 200 max but future-proof and costs nothing.

2. **Manifest write frequency during capture**
   - What we know: Writing after every frame is safest for live tail (D-13) but adds I/O
   - Recommendation: Write manifest after every frame via `persistQueue`. At 100ms intervals with ~1KB manifest updates, this is negligible I/O. Batch writes would risk data loss on crash.

3. **Startup purge behavior**
   - What we know: D-07 says purge on server start
   - Recommendation: Scan the frames directory on first `ensureLoaded()` call, remove directories older than 24h. Log what was purged. Don't block server startup -- fire-and-forget.

## Sources

### Primary (HIGH confidence)
- Codebase analysis: `src/types.ts`, `src/capture/session-manager.ts`, `src/capture/scheduler.ts`, `src/processing/grid-compiler.ts`, `src/processing/pixel-compare.ts`, `src/processing/idle-compressor.ts`, `src/profiles/profile-manager.ts`, `src/profiles/profile-resolver.ts`, `src/profiles/profile-types.ts` -- all patterns verified directly from source
- CONTEXT.md D-01 through D-25 -- all locked decisions reviewed

### Secondary (MEDIUM confidence)
- Node.js `fs/promises` API -- standard built-in, well-documented [VERIFIED: Node.js v25.8.2 runtime confirmed]

## Metadata

**Confidence breakdown:**
- Standard stack: HIGH -- no new dependencies, all existing modules verified
- Architecture: HIGH -- follows established codebase patterns (ProfileManager, SessionManager, Scheduler)
- Pitfalls: HIGH -- derived from direct codebase analysis of concurrent write and timing patterns
- Query algorithm: HIGH -- straightforward math, well-defined by CONTEXT.md decisions

**Research date:** 2026-04-13
**Valid until:** 2026-05-13 (stable domain, no external dependency churn)