# Phase 16.1: Avatar Caching Strategy Addendum — GitHub Branch Store **Appended:** 2026-03-23 **Supersedes:** The "local disk only" avatar caching recommendation in 16.1-RESEARCH.md **Confidence:** HIGH (all API behavior verified via official GitHub docs) --- ## Context The original RESEARCH.md concluded that local disk cache at `%APPDATA%/WITwhat/avatars/` was the only viable avatar storage path. This addendum evaluates a second approach raised after initial research: storing Discord avatar PNG files as committed files on a dedicated `recipient-avatars` orphan branch in the `BigscreenVR/beyond-outgoing` repository. This makes GitHub the durable store while the local disk cache becomes a performance cache rather than the primary store. --- ## Q1: What GitHub API calls are needed to commit a binary PNG file to a branch? **Answer: GitHub Contents REST API — `PUT /repos/{owner}/{repo}/contents/{path}`** This is a REST API endpoint (not GraphQL). The existing client (`GhCliProjectClient`) uses the `gh` CLI subprocess exclusively, invoking `gh api graphql ...` for all GH Project operations. For the Contents API, the same `gh` CLI can invoke REST endpoints via: ```bash gh api --method PUT repos/BigscreenVR/beyond-outgoing/contents/avatars/{discord_user_id}.png \ -F message="chore(avatars): update avatar for {discord_user_id}" \ -F branch="recipient-avatars" \ -F content="BASE64_ENCODED_PNG_BYTES" \ -F sha="CURRENT_BLOB_SHA_IF_FILE_EXISTS" ``` **Request body fields (all via `-F` in gh CLI):** | Field | Required | Type | Notes | |-------|----------|------|-------| | `message` | Yes | string | Commit message | | `content` | Yes | string | Base64-encoded file bytes — binary PNG is fine | | `branch` | No (but use it) | string | Defaults to repo default branch; specify `recipient-avatars` | | `sha` | Conditional | string | Required when updating an existing file; omit on first create | **To update an existing file, first fetch its current blob SHA:** ```bash gh api repos/BigscreenVR/beyond-outgoing/contents/avatars/{discord_user_id}.png \ --jq .sha \ -H "Accept: application/vnd.github+json" \ -F ref="recipient-avatars" ``` Wait — `gh api` for GET uses query params differently. Correct form: ```bash gh api "repos/BigscreenVR/beyond-outgoing/contents/avatars/{discord_user_id}.png?ref=recipient-avatars" \ --jq .sha ``` If the file does not yet exist this returns 404; the PUT then proceeds without a `sha` field. **From Rust**, since all GitHub operations currently go through `Command::new(&self.gh_path)`, the avatar commit uses the same subprocess pattern: ```rust // In a new AvatarBranchClient or extending GhCliProjectClient fn upload_avatar_to_branch( gh_path: &Path, owner: &str, repo: &str, discord_user_id: &str, png_bytes: &[u8], existing_sha: Option<&str>, // None = first upload, Some = update ) -> Result<(), String> { use std::process::Command; let b64 = base64_encode(png_bytes); // see note below on base64 crate let path = format!("avatars/{}.png", discord_user_id); let endpoint = format!("repos/{}/{}/contents/{}", owner, repo, path); let mut args = vec![ "api".to_string(), "--method".to_string(), "PUT".to_string(), endpoint, "-F".to_string(), format!("message=chore(avatars): update {}", discord_user_id), "-F".to_string(), format!("branch=recipient-avatars"), "-F".to_string(), format!("content={}", b64), ]; if let Some(sha) = existing_sha { args.push("-F".to_string()); args.push(format!("sha={}", sha)); } let output = Command::new(gh_path).args(&args).output() .map_err(|e| format!("gh exec: {}", e))?; if !output.status.success() { return Err(String::from_utf8_lossy(&output.stderr).to_string()); } Ok(()) } ``` **Note on base64 encoding in Rust:** The `ureq` 2.x crate does not include base64. Use the `base64` crate (v0.21 or v0.22) which is not currently in any Cargo.toml. Alternatively, use `std::fmt::Write` with manual base64 encoding, but that is hand-rolling. The `base64` crate is a single lightweight dependency. This is the **only new crate** required for the branch-store approach. ```toml # Add to crates/integrations/Cargo.toml base64 = "0.22" ``` --- ## Q2: How to create the `recipient-avatars` branch (orphan preferred) **Answer: Two REST API calls via `gh api`** An orphan branch has no parent commits and a clean, isolated history. The process uses Git's well-known empty tree SHA `4b825dc642cb6eb9a060e54bf8d69288fbee4904`: **Step 1: Create an initial commit with no parents (POST /git/commits)** ```bash gh api --method POST repos/BigscreenVR/beyond-outgoing/git/commits \ -F message="chore: initialize recipient-avatars orphan branch" \ -F tree="4b825dc642cb6eb9a060e54bf8d69288fbee4904" \ -F parents="[]" ``` Response includes `sha` of the new commit. **Step 2: Create the branch ref pointing at that commit (POST /git/refs)** ```bash gh api --method POST repos/BigscreenVR/beyond-outgoing/git/refs \ -F ref="refs/heads/recipient-avatars" \ -F sha="{SHA_FROM_STEP_1}" ``` **Idempotency:** These two steps only run once (branch setup). At runtime, the avatar upload code should first check if the branch exists (`GET /repos/.../git/ref/heads/recipient-avatars`), and only create it if the response is 404. Creating it a second time returns a 422 Unprocessable. **Rust pattern:** ```rust fn ensure_recipient_avatars_branch_exists(gh_path: &Path) -> Result<(), String> { // Check if branch exists let check = Command::new(gh_path) .args(["api", "repos/BigscreenVR/beyond-outgoing/git/ref/heads/recipient-avatars"]) .output() .map_err(|e| e.to_string())?; if check.status.success() { return Ok(()); // Branch already exists } // Step 1: Create orphan commit let commit_out = Command::new(gh_path) .args([ "api", "--method", "POST", "repos/BigscreenVR/beyond-outgoing/git/commits", "-F", "message=chore: initialize recipient-avatars orphan branch", "-F", "tree=4b825dc642cb6eb9a060e54bf8d69288fbee4904", "-F", "parents=[]", ]) .output() .map_err(|e| e.to_string())?; if !commit_out.status.success() { return Err(String::from_utf8_lossy(&commit_out.stderr).to_string()); } let commit_json: serde_json::Value = serde_json::from_slice(&commit_out.stdout) .map_err(|e| e.to_string())?; let commit_sha = commit_json["sha"].as_str().ok_or("no sha in commit response")?; // Step 2: Create ref let ref_out = Command::new(gh_path) .args([ "api", "--method", "POST", "repos/BigscreenVR/beyond-outgoing/git/refs", "-F", "ref=refs/heads/recipient-avatars", "-F", &format!("sha={}", commit_sha), ]) .output() .map_err(|e| e.to_string())?; if !ref_out.status.success() { return Err(String::from_utf8_lossy(&ref_out.stderr).to_string()); } Ok(()) } ``` **Confidence:** HIGH — verified via official GitHub REST API docs and community discussion confirming the empty tree SHA approach. The `parents=[]` / empty parents array behavior is documented in the git/commits endpoint. --- ## Q3: File naming convention **Recommendation: `avatars/{discord_user_id}.png`** This is the natural path. In the branch, the repo root contains only an `avatars/` directory. Each file is named by the Discord user's stable numeric ID (not username, which can change). ``` recipient-avatars branch: avatars/ 123456789012345678.png (discord_user_id = snowflake) 987654321098765432.png ... ``` **Why Discord user ID (snowflake), not username:** - Usernames are mutable (changed by user, resolved differently over time) - Discord user IDs are permanent numeric snowflakes - Already stored on the recipient as `discord_user_id` - The existing local cache in RESEARCH.md also uses `{discord_user_id}.png` **Path size concern:** Discord snowflake IDs are 18-digit numbers. Path becomes `avatars/123456789012345678.png` — 45 chars. Well within GitHub path limits. **Confidence:** HIGH — consistent with CONTEXT.md locked decision on local cache naming (`%APPDATA%/WITwhat/avatars/{discord_user_id}.png`) and Discord's ID stability guarantee. --- ## Q4: How to read/download avatar images from the branch during sync **Two approaches — raw URL is simpler:** ### Option A: Raw githubusercontent URL (recommended) Public repositories: `https://raw.githubusercontent.com/{owner}/{repo}/{branch}/{path}` ``` https://raw.githubusercontent.com/BigscreenVR/beyond-outgoing/recipient-avatars/avatars/{discord_user_id}.png ``` Since `BigscreenVR/beyond-outgoing` is a private repository, this URL requires authentication. The `gh` CLI auth token works for `gh api` calls but does NOT work for raw.githubusercontent.com requests without an explicit bearer token header. **For private repos, use the Contents API instead:** ### Option B: Contents API with `?ref=recipient-avatars` (correct for private repo) ```bash gh api "repos/BigscreenVR/beyond-outgoing/contents/avatars/{discord_user_id}.png?ref=recipient-avatars" ``` The response JSON contains: - `content`: base64-encoded PNG bytes - `sha`: current blob SHA (needed for future updates) - `download_url`: a time-limited authenticated URL (not stable for caching) In Rust: ```rust fn fetch_avatar_from_branch( gh_path: &Path, discord_user_id: &str, ) -> Result<(Vec, String), String> { // Returns (png_bytes, blob_sha) let endpoint = format!( "repos/BigscreenVR/beyond-outgoing/contents/avatars/{}.png?ref=recipient-avatars", discord_user_id ); let output = Command::new(gh_path) .args(["api", &endpoint]) .output() .map_err(|e| e.to_string())?; if !output.status.success() { return Err(String::from_utf8_lossy(&output.stderr).to_string()); } let json: serde_json::Value = serde_json::from_slice(&output.stdout) .map_err(|e| e.to_string())?; let b64 = json["content"].as_str().ok_or("no content field")?; // GitHub base64 response includes newlines; strip them let b64_clean = b64.replace('\n', ""); let bytes = base64_decode(&b64_clean).map_err(|e| e.to_string())?; let sha = json["sha"].as_str().unwrap_or("").to_string(); Ok((bytes, sha)) } ``` **Important:** Store the returned `sha` alongside the local cached file (e.g., in SQLite as `avatar_gh_sha`). This SHA is required when calling the PUT endpoint to update an existing file. Without the current SHA, the update call returns 409 Conflict. **Confidence:** HIGH — Contents API `?ref=` parameter behavior verified via official GitHub docs. --- ## Q5: Size and rate limit considerations **Avatar file size:** Discord avatars at `?size=256` are typically 20-80 KB PNG. At 100 recipients, total branch storage is approximately 2-8 MB — negligible. **GitHub API rate limit:** Authenticated requests are limited to 5,000 requests/hour for personal access tokens (verified via GitHub REST API docs 2025). The `gh` CLI authenticates via the stored PAT. **Request budget for 100 recipients:** | Operation | Requests | Frequency | |-----------|----------|-----------| | Branch existence check | 1 | On startup | | Avatar existence check (fetch sha) | 1 per recipient | Sync cycle | | Avatar download (fetch content) | 1 per recipient needing refresh | Sync cycle | | Avatar upload (PUT) | 1 per new/changed avatar | Sync cycle | For a full initial sync of 100 recipients: up to ~300 requests (check + download + upload per recipient). At 5,000 requests/hour limit, this is 6% of the hourly budget. Incremental syncs (only changed avatars) cost far less. **Staleness strategy:** Compare `avatar_gh_sha` in SQLite against the `sha` returned by the contents fetch. If identical, skip download. This reduces sync cost to ~100 requests (one HEAD- equivalent GET per recipient) for steady-state operation. **Practical constraint:** The GH CLI `gh api` calls are synchronous subprocess invocations. 100 sequential `gh api` calls during sync will be slow (~1-3 seconds per call including process spawn overhead). Recommend batching avatar operations asynchronously or running them after the main sync on a background thread. **Confidence:** HIGH for rate limits (official docs). MEDIUM for per-call latency estimate (based on typical `gh api` subprocess overhead observed in existing code). --- ## Q6: Local cache story with GitHub as durable store **Two-tier caching:** | Tier | Location | Role | Lifetime | |------|----------|------|---------| | GitHub branch | `BigscreenVR/beyond-outgoing:recipient-avatars` | Durable store — survives local wipe, works across machines | Permanent until manually deleted | | Local disk | `%APPDATA%/WITwhat/avatars/{discord_user_id}.png` | Performance cache — instant display without API call | Until app cache is cleared or avatar refresh triggered | **Sync flow:** 1. App starts — load avatar images from local disk cache (instant, no network) 2. Background sync cycle runs — for each recipient: a. Check if local cache file exists and `avatar_gh_sha` in SQLite matches branch b. If match: skip (local cache is current) c. If mismatch or no local file: download from branch, write to local disk, update SQLite sha 3. When Discord username is edited and avatar re-fetched from Discord CDN: a. Write new PNG to local disk (immediate) b. Upload to GitHub branch (background, fire-and-forget) c. Update `avatar_gh_sha` in SQLite **Local cache still needed for:** Fast startup display (no network required), offline mode, and avoiding per-display-cycle API calls. GitHub branch is the recovery/portability layer, not the hot path. **New SQLite column required:** `avatar_gh_sha TEXT` on the recipients table (added in V003 migration alongside `avatar_hash`). Stores the GitHub blob SHA for the uploaded avatar, enabling skip-if-current logic during sync. --- ## Q7: Reusable auth/client code **The existing `GhCliProjectClient` pattern is directly reusable.** The `gh` CLI subprocess approach handles authentication transparently — `gh` reads the stored PAT from its own credential store (configured once via `gh auth login`). No token is passed explicitly to `Command::new(&self.gh_path)` calls; the `gh` CLI reads it from its own keyring. This means: - No new auth needed - No PAT handling in app code for the branch operations - The same `gh` binary found by `find_gh()` supports both GraphQL (`gh api graphql`) and REST (`gh api --method PUT repos/...`) - Error detection via `detect_auth_error(&output.stderr)` works identically for REST calls **New struct recommended:** Rather than extending `GhCliProjectClient` (which is GH Project V2-specific), create a sibling struct in `crates/integrations/src/github/`: ``` crates/integrations/src/github/ ├── avatar_branch_client.rs # NEW — REST-based branch file operations └── gh_cli_client.rs # EXISTING — GraphQL-based GH Project V2 operations ``` `AvatarBranchClient` wraps the same `gh_path: PathBuf` and follows the same `Command::new(gh_path).args([...]).output()` pattern. It is NOT a `GithubProjectClient` trait implementor — it is a separate struct with its own methods. **Token scope:** The GH PAT already used for GH Project access requires `project` scope. The Contents API (`PUT /repos/.../contents/...`) and git refs API require `repo` scope (or `public_repo` for public repos). Since `BigscreenVR/beyond-outgoing` is private, the PAT must have `repo` scope. **This must be verified against the actual PAT configuration.** If the existing PAT was created with only `project` scope, this approach requires a PAT scope expansion. **Confidence:** HIGH for the `gh` CLI subprocess pattern reuse. MEDIUM for PAT scope assumption — requires verification that the existing PAT has `repo` scope, not just `project`. --- ## Architecture Decision: Branch Store vs Local-Only | Dimension | Local-Only (RESEARCH.md) | Branch Store + Local Cache (this addendum) | |-----------|--------------------------|---------------------------------------------| | Implementation cost | Lower — no new GH API calls | Higher — new client, base64 dep, SHA tracking | | Durability | Lost on local wipe | Survives wipe; recoverable from GH | | Cross-machine | Not shared | Shared automatically on sync | | Offline display | Works (cached on disk) | Works (cached on disk) | | New dependencies | None | `base64` crate (lightweight) | | PAT scope risk | None | Requires `repo` scope on PAT | | Sync complexity | Simple | Requires SHA tracking, two-tier logic | | GitHub API calls | None | ~1-3 per recipient per sync cycle | **Recommendation for planner:** The branch-store approach is architecturally cleaner for a multi-machine team workflow but adds meaningful implementation complexity. The local-only approach is lower risk and sufficient if avatars are acceptable to re-fetch after a machine wipe. The decision should be made before the plan is written, as it affects: - Whether `base64` crate is added - Whether `avatar_gh_sha` column is in V003 - Whether `AvatarBranchClient` is in scope for this phase --- ## New Pitfalls (branch-store approach only) ### Pitfall 9: Missing `sha` on file update returns 409 **What goes wrong:** Calling PUT on an existing file without the current blob SHA returns HTTP 409 Conflict. The file is not updated. **How to avoid:** Always GET the file's SHA before updating. Store SHA in SQLite. On 404 from GET, skip SHA (first upload). On any other GET error, abort the upload rather than potentially overwriting with wrong SHA. ### Pitfall 10: gh CLI base64 output includes newlines **What goes wrong:** The GitHub Contents API returns base64-encoded content with `\n` characters every 60-76 chars (standard PEM-style line breaks). Passing this directly to a base64 decoder that expects unpadded continuous base64 will fail. **How to avoid:** Strip all `\n` from the content string before decoding: `b64.replace('\n', "")`. ### Pitfall 11: PAT scope insufficient for Contents API **What goes wrong:** The existing PAT may have been scoped to `project` only (for GH Project V2 GraphQL mutations). The Contents API requires `repo` (or `contents:write` for fine-grained PATs). **How to avoid:** Before implementing, check the PAT scope. If insufficient, require the user to regenerate/expand their PAT. Document this as a setup step. ### Pitfall 12: Orphan branch creation races on first launch **What goes wrong:** Two simultaneous app instances both detect the branch as missing and both try to create it. The second POST to `/git/refs` returns 422 (ref already exists). **How to avoid:** Treat 422 on branch creation as a non-error (idempotent success). The ref was created by the other instance. --- ## Summary The GitHub branch store approach is viable and uses only the existing `gh` CLI mechanism. The required API calls are: 1. **Branch setup (once):** `POST /git/commits` (empty tree, no parents) + `POST /git/refs` 2. **Avatar upload:** `GET contents/avatars/{id}.png?ref=recipient-avatars` (for sha) + `PUT contents/avatars/{id}.png` (with base64 content) 3. **Avatar download:** `GET contents/avatars/{id}.png?ref=recipient-avatars` (base64 decode response) All three use the `gh api` subprocess pattern already established in `GhCliProjectClient`. The only new dependency is a base64 crate. The main risk is PAT scope — must have `repo` scope for the Contents API on a private repository. --- ## Sources ### Primary (HIGH confidence) - [GitHub REST API — Create or update file contents](https://docs.github.com/en/rest/repos/contents) — PUT endpoint fields, branch param, sha requirement, base64 encoding - [GitHub REST API — Rate limits](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api) — 5,000 req/hour for authenticated PAT - [GitHub community discussion #24699](https://github.com/orgs/community/discussions/24699) — orphan branch via REST API using empty tree SHA `4b825dc642cb6eb9a060e54bf8d69288fbee4904` - Codebase read: `crates/integrations/src/github/gh_cli_client.rs` — confirms `gh api` subprocess pattern, same approach applicable to REST endpoints ### Secondary (MEDIUM confidence) - [TIL: Creating files via GitHub API and CLI](https://www.zufallsheld.de/2023/12/11/til-how-to-create-github-files-via-api/) — confirms `gh api --method PUT` with `-F content=BASE64` works for binary file upload **Research date:** 2026-03-23 **Valid until:** 2026-05-23 (GitHub REST API is stable; rate limits unchanged since 2021)