314 lines
24 KiB
Text
314 lines
24 KiB
Text
# progress.txt — directus-extension-generate-image-previews
|
||
|
||
Append-only log of decisions, evidence, and learnings. Newest at the bottom.
|
||
|
||
## 2026-10-25 — project scaffolded
|
||
|
||
- Opened this standalone extension project to pre-generate Directus image previews.
|
||
- Source-verified the strategy against Directus **v11.17.4** (see `.opencode/goal.md`).
|
||
- Root causes why existing/community extensions do not work:
|
||
1. `resolvePreset` never reads `key` → `{transformationParams:{key}}` yields `transforms=[]` → no-op.
|
||
2. `system-*` presets are `format:'auto'` → `jpg`/`png` server-side vs `avif` in-browser → different hash.
|
||
3. "undrained stream truncates file" — false (write is awaited).
|
||
4. admin uses `?key=system-*`, not raw width/height.
|
||
- Admin URLs (v11.17.4 app source): files grid `?key=system-medium-cover&v=<modified_on>`;
|
||
image field `system-large-cover|contain`; image display `system-small-cover`.
|
||
- Server bottleneck evidence (2026-10-23 log triage): `storage:local:responseTime 1206ms`,
|
||
`pg:responseTime` 150–738ms; no 503s, no sharp errors.
|
||
|
||
## 2026-10-25 — scope expanded after user direction (v2)
|
||
|
||
- **No token needed.** Dropped HTTP-replay as primary. Strategy: **in-process**
|
||
`AssetsService.getAsset(id, { transformationParams: <full preset object>, acceptFormat: 'avif'|'webp' })`.
|
||
No HTTP → no token, no proxy, no rate limiter. (HTTP-replay kept as documented fallback.)
|
||
- **Bundle extension**: api `hook` + api `endpoint` + app `module`.
|
||
- **Presets: all, dynamically** — system `SYSTEM_ASSET_ALLOW_LIST` (6) + `directus_settings.storage_asset_presets`,
|
||
controller order `[...SYSTEM, ...custom]`, first-match-wins; new presets picked up per run.
|
||
- **Admin module**: progress bar, disk usage (`list`+`stat`), config form, diagnostics panel.
|
||
- **Config**: maintenance window + concurrency (throttle knob; in-process bypasses HTTP rate limiter),
|
||
persisted in JSON.
|
||
- Confirmed `acceptFormat:'avif'` in a direct call equals browser `Accept: image/avif`.
|
||
|
||
## 2026-10-25 — diagnostics + safety + autonomy (v3)
|
||
|
||
- **Diagnostics / manual generation** (F06): `POST /generate` (one file, dry-run supported),
|
||
`GET /resolve-plan` (resolved transforms + predicted filename, no write), `GET /ops` (ring buffer).
|
||
- **Safe testing** (F02): staging compose stack + deploy/rollback scripts; fail-open, `enabled` default false.
|
||
- **Autonomous remote feedback**: staging-only, least privilege (scoped token, no prod Docker socket,
|
||
restricted SSH, private network, human-only prod promotion).
|
||
|
||
## 2026-10-25 — implementation + local verification (autonomous session)
|
||
|
||
Built the full extension locally and smoke-tested against an **isolated** Directus 11.17.4 stack
|
||
(`dev/compose.dev.yml`, port 8057, own pg + volumes). No contact with the prod instance.
|
||
|
||
### Implemented
|
||
- `src/lib/`: settings (env + JSON), state (`state.jsonl`, run state, lock, cancel flag),
|
||
presets (pinned system copy + custom merge, controller parity), warmer (in-process `getAsset` with
|
||
`acceptFormat` variants, concurrency, per-task error isolation, stream release), window, storage
|
||
(dynamic `getStorage` + local-FS fallback), diagnostics (resolve-plan prediction + ops ring buffer),
|
||
backfill (paginated, resumable, window-aware, idempotent).
|
||
- `src/endpoint/index.ts`: `/status`, `/config` (GET/PUT), `/presets`, `/run`, `/run/cancel`,
|
||
`/disk-usage`, `/generate`, `/resolve-plan`, `/ops`; admin-only guard.
|
||
- `src/hook/index.ts`: `files.upload` → fire-and-forget warm.
|
||
- `src/module/`: admin module (progress bar, disk usage, config form, diagnostics panel, ops table).
|
||
- `dev/`: compose stack, `sync-extension.sh`, seed images.
|
||
|
||
### Bugs found & fixed during smoke test
|
||
1. **Directus coerces env values** (`"true"` → boolean `true`, `"1"` → number). My string-only parsers
|
||
threw `e.toLowerCase is not a function`, which silently broke the upload hook. Fixed `settings.ts`
|
||
to accept `unknown` and stringify. Regression test added.
|
||
2. **State dir EACCES.** `/directus/extensions/.image-previews` (named volume on a non-existent path)
|
||
is root-owned; the API runs as `node`. This broke config writes and the backfill lock. Fixed: default
|
||
state dir is now `<STORAGE_LOCAL_ROOT>/.image-previews` (i.e. `/directus/uploads/.image-previews`),
|
||
which is writable; `IMAGE_PREVIEWS_STATE_DIR` still overrides.
|
||
3. **Bind-mount reload gotcha.** `rm -rf` of the bind-mounted extension dir breaks the Docker mount
|
||
(Directus removes the extension and never re-adds it). `sync-extension.sh` now updates in place;
|
||
restart the dev container after a sync.
|
||
|
||
### Verified evidence (local stack)
|
||
- `npm run typecheck` → 0 errors. `npm test` → **32/32 pass**. `npm run build` → `dist/app.js` + `dist/api.js`.
|
||
- Directus logs: `Loaded extensions: directus-extension-generate-image-previews` (no errors).
|
||
- `/status` → `{enabled:true, inWindow:true, locked:false}`; `/presets` → 6 system presets.
|
||
- Upload a JPEG → hook warmed **12 files** (6 presets × avif/webp) in `/directus/uploads`.
|
||
- **HASH-MATCH GATE PASSED:** after deleting the cached files, a real
|
||
`GET /assets/<id>?key=system-medium-cover` with `Accept: image/avif` produced exactly
|
||
`<id>__f15aa4b815722867859d2a3fa46ab50834dfc8dc.avif` — byte-identical to `resolve-plan`'s prediction.
|
||
- `/generate` dry-run reports `exists:true/false` correctly; `/ops` shows per-task timing.
|
||
- Backfill: `total=36` (3 images × 6 presets × 2 formats), `generated=36`, state `state.jsonl` = 36 lines.
|
||
Second run: `generated=0, hits=36`, state unchanged, file count unchanged → **idempotent/resumable**.
|
||
- Window gating: `/run` outside `03:00–05:00` → `409 outside-window` with `nextWindow`.
|
||
- `/disk-usage` → `157633 bytes / 36 files (source: storage)`.
|
||
- Cold vs warm (`system-medium-cover`): 114 ms → 16–30 ms. (`system-large-cover`): 304 ms → 36 ms.
|
||
|
||
### Decisions recorded
|
||
- System presets: version-pinned copy in `src/lib/system-presets.ts` (verbatim from `api/src/constants.ts`
|
||
v11.17.4). No `@directus/api` import needed for presets.
|
||
- Storage/disk usage: `@directus/api/storage/index` loaded via dynamic import (hidden from the bundler),
|
||
with a local-FS fallback; shares the host `getStorage()` singleton.
|
||
- New-preset trigger: **manual** via `POST /run` (idempotent) + the module's Run button; no
|
||
`settings.update` auto-trigger (avoids heavy background runs on unrelated settings changes).
|
||
- Scheduling: external cron → `POST /image-previews/run` (default).
|
||
- State/config dir: `<STORAGE_LOCAL_ROOT>/.image-previews` (writable), override with `IMAGE_PREVIEWS_STATE_DIR`.
|
||
|
||
### Additional verification (same session)
|
||
- `GET /extensions` → bundle registered with entries `module`, `hook`, `endpoint`, all enabled.
|
||
- Inert test: with `enabled:false`, uploading an image produced **zero** new preview files (25 → 25).
|
||
- Added `dev/compose.staging.yml`, `dev/deploy-staging.sh`, `dev/rollback-staging.sh` (templates;
|
||
NOT executed against any remote host).
|
||
- Added `storage.test.ts`; total tests now **34/34**.
|
||
- Restarted the dev stack after the final build; `/status`, `/disk-usage`, and bundle entries all good.
|
||
|
||
### Still open
|
||
- Admin module UI not yet verified in a real browser (bundle + entries registered; rendering/polling untested).
|
||
- Cancel + concurrent-run 409 not exercised end-to-end (unit-tested only; local runs finish too fast).
|
||
- Staging deploy not executed (scripts/templates provided); prod rollout not done.
|
||
- F01/F02/F07/F10 remain `passes:false` (F01 publish step, F02 staging run, F07 browser, F10 rollout).
|
||
|
||
## 2026-10-25 — feedback round (UI, defaults, custom formats, docs)
|
||
|
||
- **Defaults changed**: `enabled` now `true`; maintenance window defaults to `01:00–04:00`.
|
||
- **Custom fixed-format presets verified**: `custom-png` (format `png`) generated `.png` and
|
||
`resolve-plan` predicted the exact filename (`exists:true`). `custom-auto` generated avif+webp.
|
||
An unsupported preset (`format: bmp`) failed with Directus's clear error
|
||
("Expected one of: heic, heif, avif, jpeg, … received bmp") and is surfaced in Recent operations / `/ops`.
|
||
- **Effective format fix**: ops/presets now report the preset's real output format (`png`) instead of
|
||
`null`; backfill resume keys use the effective format (fixed a re-warm mismatch).
|
||
- **Progress accounting fix**: already-cached tasks now count toward `done`/`hits`. Runs report
|
||
`128/128` (120 hits + 8 expected failures) instead of `8/128`. After removing the unsupported preset:
|
||
`120/120 hits, 0 failed, 0 generated`.
|
||
- **Storage reporting**: `/disk-usage` + panel now split **previews** vs **originals** vs **total**
|
||
(raw + human): e.g. previews 774.7 KB / 115 files, originals 24.9 MB / 9 files, total 25.6 MB / 124.
|
||
- **Human-readable fields** added to `/status` (ISO + human dates, `etaHuman`, `rateHuman`),
|
||
`/disk-usage` (`bytesHuman`), `/ops` (`tsHuman`, `msHuman`), `/generate` (`msHuman`).
|
||
- **Admin module rewritten with Directus UI kit** (`v-button`, `v-input`, `v-checkbox`, `v-select`,
|
||
`v-notice`, `v-progress-linear`), field explanations, preset table with effective formats + shadow
|
||
warnings, storage table, error column in Recent operations, and a full collapsible **API reference**.
|
||
- **Run now** now ignores the maintenance window; **Cancel** only renders while a run is active.
|
||
- **Dimension vs file size clarified** in README/goal/AGENTS: the 6000 limit is a pixel-dimension guard in
|
||
Directus, not a file-size limit; the extension imposes none.
|
||
- Tests: **40/40** pass; typecheck clean; build OK.
|
||
|
||
### Still open
|
||
- Admin module UI not yet verified in a real browser (bundle + entries registered; rendering/polling untested).
|
||
- Cancel + concurrent-run 409 not exercised end-to-end.
|
||
- Staging deploy not executed (scripts/templates provided); prod rollout not done.
|
||
- F01/F02/F07/F10 remain `passes:false`.
|
||
|
||
## 2026-10-25 — production deploy + accuracy fixes + in-process scheduler
|
||
|
||
- Deployed to prod via local build + SCP archive (prod has no Node). A source clone is kept outside the
|
||
watched extensions directory.
|
||
- Verified: bundle loaded on all 4 PM2 workers, endpoints mounted (403 admin guard), no errors.
|
||
- **Backfill counting fix**: pre-scan storage filenames; each task's predicted cache filename is checked
|
||
before generating. Existing previews count as `already cached` and are skipped. (Commit 18db52a.)
|
||
- **In-process nightly scheduler**: hook registers `schedule('*/15 * * * *')`; runs only inside the
|
||
maintenance window, no external cron/token, pauses when the window closes and resumes next night.
|
||
- UI now shows last-run errors and `paused (window closed)`.
|
||
- Tests 43/43, typecheck clean.
|
||
|
||
## 2026-10-30 — file-library ordering / TUS `uploaded_on` / failure observability
|
||
|
||
- User report: fresh files absent from folder list unless filtered by `uploaded_on`; 4,649-item Documents
|
||
folder; page size raised to 1000 remained slow.
|
||
- Directus 11.17.4 source check: filtered result is an `_and` subset of unfiltered; no app item cache.
|
||
Postgres `ORDER BY uploaded_on DESC` = NULLS FIRST, so a large null-date block fills every page before
|
||
fresh rows. TUS (`TusDataStore.create` + `onUploadFinish`) never sets `uploaded_on`; regular
|
||
`POST /files` does. Upstream #28042 fixed only in 12.4.0; host stays 11.17.4.
|
||
- Host checks (pgAdmin): `created_on IS NULL` = **0**; `directus_files` total **39,350**,
|
||
`uploaded_on IS NULL` = **2,836**, dated = **36,514**.
|
||
- `uploaded_on` default added successfully: `DEFAULT (now() AT TIME ZONE 'utc')`.
|
||
- **Backfill UPDATE was not applied**: `uploaded_on IS NULL` still 2,836. Defaults do not backfill
|
||
history. Re-run `UPDATE directus_files SET uploaded_on = created_on WHERE uploaded_on IS NULL;`
|
||
(transaction, backup) and verify null count = 0.
|
||
- Slow thumbnails: backfill only ~95,744 / 661,232 tasks done (~14%); loading 1,000 cards issues 1,000
|
||
transform requests, most cold → on-demand sharp. Use 25–100 page size; let backfill finish or run
|
||
outside-window with low concurrency.
|
||
- Added feature specs F11 (durable `failures.jsonl` + bounded retry + `/failures` + module table) and
|
||
F12 (host `uploaded_on` remediation). F12 remains `passes:false` until the backfill UPDATE is
|
||
actually applied and one new TUS upload is verified.
|
||
|
||
## 2026-10-30 — production outage: uncommitted pgAdmin DDL held lock (resolved)
|
||
|
||
- Symptom: Directus UI/API slow, 500s, health timeouts, container unhealthy. `directus.log` showed
|
||
repeated `KnexTimeoutError: Timeout acquiring a connection`, `Can't connect to the database`, and
|
||
60s request aborts.
|
||
- Root cause: pgAdmin backend **PID 674331** (`pgAdmin 4 - CONN:4204762`) was `idle in transaction`
|
||
holding a granted `AccessExclusiveLock` on `public.directus_files`. Its last query was the
|
||
`ALTER TABLE ... ALTER COLUMN uploaded_on SET DEFAULT ...`; it was never committed. All 80 Directus
|
||
pool connections (4 workers × 20) were active waiting on `Lock/relation` for `directus_files`, so the
|
||
API pool was exhausted.
|
||
- A `commit;` from a different pgAdmin connection (PID 674244) does not commit another session's
|
||
transaction. DDL must be run with auto-commit on or `BEGIN; ...; COMMIT;` in the same session.
|
||
- Recovery: terminated PID 674331 (`pg_terminate_backend`). Its uncommitted ALTER rolled back; lock
|
||
released; Directus pool drained; `/server/health` returned `{"status":"ok"}` and container health
|
||
returned `healthy`.
|
||
- Re-applied default safely with `lock_timeout=3s`:
|
||
`DEFAULT (now() AT TIME ZONE 'utc')`. Verified in `information_schema.columns`.
|
||
- Applied backfill in one transaction with `lock_timeout=5s`, `statement_timeout=30s`:
|
||
`UPDATE directus_files SET uploaded_on = created_on WHERE uploaded_on IS NULL;` → **UPDATE 2836**,
|
||
COMMIT. Verified `null_dates = 0`, `dated = 39,350`, zero `Lock` waiters afterwards.
|
||
- No code/config deploy involved. Raw SQL did not touch `modified_on`/`modified_by`.
|
||
- Remaining verification: upload one TUS file and confirm non-null `uploaded_on` + `tus_id = null`;
|
||
then confirm the folder list shows it at top with `-uploaded_on`.
|
||
- Verified on 2026-10-30 after recovery:
|
||
- latest 5 TUS uploads all have non-null `uploaded_on`, `tus_id = null`, correct `folder`
|
||
- `uploaded_on - created_on` ≈ 0–0.24s (default is applied at INSERT, finish is immediate)
|
||
- `show timezone` = `Etc/UTC`, so `(now() AT TIME ZONE 'utc')` cast to `timestamptz` is correct
|
||
- zero rows `state = 'idle in transaction'`; no lock waiters
|
||
- DB/TUS acceptance for F12 is done. Only remaining check: browser Files view shows a fresh TUS file
|
||
at the top of the unfiltered folder with `sort=-uploaded_on` and `page=1`.
|
||
- F12 passes. Browser shows fresh files at top when the cards sort control sends `-uploaded_on`.
|
||
- Apparent "inverted sorting" is a Directus cards UI icon bug, not data. v11.17.4
|
||
`app/src/layouts/cards/components/header.vue` flips the `sort` glyph for descending, while tabular
|
||
flips it for ascending. Query semantics are correct: `sort[]=uploaded_on` = ascending (oldest first),
|
||
`sort[]=-uploaded_on` = descending (newest first). Selecting a field resets direction to ascending.
|
||
Workaround: toggle until DevTools shows `-uploaded_on`, ignore glyph; or use tabular.
|
||
|
||
## 2026-10-30 — F11 persistent failure journal + bounded retry + /failures + module table
|
||
|
||
Applicant: coder
|
||
Feature: F11 — Persistent failure journal, bounded retry, failure API and module view
|
||
Branch: feat/F11 @ de25e50 (base goal/image-previews @ 7951f82)
|
||
|
||
RUN: 2026-10-30T06:26:28Z | coder | /home/rawman/Projects/DirectusExtensions/directus-extension-generate-image-previews | feat/F11 @ de25e50 | npx tsc --noEmit; npm test; npm run build | specs=0 | exit=0 | pass=55 | log=.opencode/tmp/runs/f11-test.log
|
||
|
||
Summary:
|
||
- Added `src/lib/failures.ts`: append-only `<state-dir>/failures.jsonl`, replay of the latest record per
|
||
`(file_id,preset_key,format)` (key = `completedKeyOf`), corrupt-line tolerance, atomic compaction
|
||
(unresolved + last 1,000 recovered), and `listFailures` with status filter + limit/offset.
|
||
- Added `src/lib/retry.ts`: MAX_ATTEMPTS=3, 5m/15m backoff, `nextRetryAt`, `dueRetries`,
|
||
`journalFailure`/`markRecovered`, and `runDueRetries` (enabled/window/lock gated, targeted bounded
|
||
batch default 50, grouped by file, full preset + acceptFormat via `warmVariant`, completion + recovered
|
||
on success, attempt+1 + next retry on failure, compaction under `run.lock`, never throws).
|
||
- Hooked failure journaling into the upload warm hook, `POST /generate`, and the backfill; successes mark
|
||
prior failures recovered. Journal writes never alter responses or hook behavior.
|
||
- Hook scheduler tick now runs due retries first, then the existing backfill.
|
||
- Added admin-only `GET /image-previews/failures?status=&limit=&offset=` with one batched
|
||
`directus_files` metadata query (`filename_download`, `type`, `folder`) and ISO/human timestamps.
|
||
- Module: “Failed previews” table + Refresh (no toasts); API reference includes `/failures`; polling
|
||
interval unchanged.
|
||
- Types added; README/AGENTS/project SKILL state-file docs updated.
|
||
- Tests: `failures.test.ts` (append/replay/latest-state/corrupt/idempotent/compaction/list) and
|
||
`retry.test.ts` (backoff/exhaustion/due selection/bounded batch/grouping/missing preset/recovered/
|
||
failure/disabled).
|
||
|
||
Files changed:
|
||
- src/lib/failures.ts (new), src/lib/failures.test.ts (new)
|
||
- src/lib/retry.ts (new), src/lib/retry.test.ts (new)
|
||
- src/lib/types.ts, src/lib/warmer.ts, src/lib/backfill.ts
|
||
- src/hook/index.ts, src/endpoint/index.ts, src/module/module.vue
|
||
- README.md, AGENTS.md, .opencode/skills/directus-image-previews/SKILL.md, .gitignore
|
||
- .opencode/progress.txt
|
||
|
||
Commands:
|
||
- `npx tsc --noEmit` → exit 0, 0 errors (`.opencode/tmp/runs/f11-tsc.log`)
|
||
- `npm test` → exit 0, 55/55 pass (`.opencode/tmp/runs/f11-test.log`)
|
||
- `npm run build` → exit 0, `dist/app.js` + `dist/api.js` emitted (`.opencode/tmp/runs/f11-build.log`)
|
||
|
||
Manual verification steps:
|
||
1. Cause a failure (unsupported preset format or missing file) from upload, `POST /generate`, and backfill;
|
||
confirm one line per failure in `<state-dir>/failures.jsonl` and a warn log with `fileId`,
|
||
`preset_key`, `format`, `attempt`, `next_retry_at`, `error`.
|
||
2. Inside the maintenance window with no backfill active, wait for the next 15-minute tick; confirm a due
|
||
pair is retried, success writes `state.jsonl` + a `recovered` event and logs info, failure writes
|
||
`attempt+1` with 15m backoff (or exhausted at 3).
|
||
3. `GET /image-previews/failures?status=failed&limit=50&offset=0` as admin: records include file metadata,
|
||
attempts, first/last failure, next retry raw+ISO/human, status, error, ms; non-admin gets 403.
|
||
4. Module “Failed previews” table renders and Refresh reloads; no toast/notification appears.
|
||
|
||
Not verified locally:
|
||
- Browser rendering of the new module table (no live Directus/admin session in this run).
|
||
- Live PM2 `schedule('*/15 * * * *')` tick and maintenance-window retry execution.
|
||
- Concurrent PM2 append behavior and compaction race with upload-hook appends.
|
||
|
||
## 2026-10-30 — F11 review fixes
|
||
|
||
- Head: `feat/F11` @ `e10819c`.
|
||
- Replay now derives attempts from `event.attempt`; fail→recover→fail starts a fresh cycle (attempt 1) and duplicate failed events are idempotent.
|
||
- Journal writer lock (`failures.lock`, 20×10 ms, 30 s stale): appends serialize, compaction returns `false` if it cannot lock, and append still falls back to writing.
|
||
- Backfill loads the journal once, skips unresolved failures per pair, and caps journal attempts at `MAX_ATTEMPTS`; recovered pairs still append `recovered`.
|
||
- Added `/failures` shape/validation tests, retry lock/window tests, fail→recover→fail, attempt cap, and 50-append compaction race tests.
|
||
|
||
## 2026-10-30 — F11 second review fixes
|
||
|
||
- Implementation head: `feat/F11` @ `7f46493` (progress commit follows).
|
||
- Journal writes now require the `failures.lock` directory (`owner.json` with random token + pid) and never append unlocked. Blocking acquisition defaults to 10 s / 10 ms poll; timeout drops the event with `image-previews: could not journal failure` and `fileId`/`preset_key`/`format`.
|
||
- Stale locks are broken only when the owner pid is dead (`process.kill(pid, 0)` → `ESRCH`); mtime staleness removed. Release only removes the lock when `owner.json.token` still matches.
|
||
- `compactFailures` uses the same lock across read→write temp→rename, returns `false` when it cannot acquire, and uses a pid+token-unique temp filename.
|
||
- Tests now cover live-owner append refusal, dead-pid recovery, foreign-token release, live-lock compaction refusal, temporarily-held-lock 50-append race, and `planFileTasks` total/resume/skip counting.
|
||
- `npx tsc --noEmit` 0 errors; `npm test` 69/69 pass; `npm run build` emits both bundles. Logs: `.opencode/tmp/runs/f11-fix2-{tsc,test,build}.log`.
|
||
- Remaining manual checks: live PM2 scheduler retry tick, browser module rendering, and a real multi-worker compaction/append run.
|
||
|
||
## 2026-10-30 — F11 final lock hardening
|
||
|
||
- Implementation head: `feat/F11` @ `bc20c6a` (progress commit follows).
|
||
- Lock is now an atomic `failures.lock` file written with `flag:'wx'`; owner carries `token`, `pid`, `bootId`, `startTime`, `ts`. Identity readers use `/proc/sys/kernel/random/boot_id` and `/proc/<pid>/stat` field 22, with injectable readers for tests.
|
||
- Dead-lock reclaim runs only under `failures.lock.reclaim` (mkdir guard). Inside the guard it re-stats/re-reads the current owner and unlinks only a dead owner or an unreadable owner older than the 2 s grace. Live owners are never broken; stale guards older than 30 s are removed.
|
||
- Release unlinks only when the current `owner.token` matches; no unlink on mismatch or read error.
|
||
- Tests added: concurrent reclaim of a dead lock, missing/corrupt owner grace, PID reuse by bootId and startTime, guard-held reclaim and stale-guard recovery. Existing live-owner timeout, foreign-token release, 50-append race, and compaction-blocked tests stay green.
|
||
- `npx tsc --noEmit` 0 errors; `npm test` 76/76 pass; `npm run build` emits both bundles. Logs: `.opencode/tmp/runs/f11-lock-{test,build}.log` and `f11-lock-final-*`.
|
||
- Remaining manual checks: live PM2 multi-worker append/compaction, live scheduler retry tick, browser module rendering.
|
||
|
||
## 2026-10-30 — F11 reclaim-guard lifecycle fix
|
||
|
||
- Implementation head: `feat/F11` @ `cd973ca` (progress commit follows).
|
||
- Stale guard removal is now atomic: `rename(guardPath, guardPath.stale.<staleKey>)` with a marker inside the quarantined dir; losers retry instead of deleting a successor guard. Guard writes `owner.json` `{token,pid,ts}` and its `finally` only removes the guard when the token matches.
|
||
- Added injectable `now` clock. Tests: two waiters on one stale guard (both events survive), foreign-token guard release keeps the successor, plus existing lock/race/compaction tests.
|
||
- `npx tsc --noEmit` 0; `npm test` 78/78 pass; `npm run build` emits both bundles. Logs: `.opencode/tmp/runs/f11-guard2-{test,build}.log` and `f11-guard-final-*`.
|
||
- Remaining manual checks: live PM2 multi-worker append/compaction, live scheduler retry tick, browser module rendering.
|
||
|
||
## 2026-10-30 — F11 stale-guard snapshot verification
|
||
|
||
- Implementation head: `feat/F11` @ `4bd69dd` (progress commit follows).
|
||
- Stale-guard branch now snapshots `observedMtime` + `observedOwner` before any re-read, quarantines with `guardPath.stale.<pid>.<uuid>`, then verifies the moved directory's owner token (or both null) and mtime against the snapshot. Mismatch restores the successor guard and retries; restore failure returns false without running the action.
|
||
- Added deterministic successor-replacement test via guard hooks; both appends survive and the successor guard is restored.
|
||
- `npx tsc --noEmit` 0; `npm test` 79/79 pass; `npm run build` emits both bundles. Logs: `.opencode/tmp/runs/f11-guard3-{test,build}.log` and `f11-guard3-final-*`.
|
||
- Remaining manual checks: live PM2 multi-worker append/compaction, live scheduler retry tick, browser module rendering.
|
||
|
||
## 2026-10-30 — Manager: F11 accepted, F12 host fix recorded
|
||
|
||
- Decision: squash-merged `feat/F11` into `goal/image-previews` after reviewer APPROVE (79/79 tests, tsc clean, build OK) and flipped F11 passes=true. User waived the manual browser/live-PM2 checks; recorded as unverified in the F11 note.
|
||
- Context: three review rounds hardened replay attempts, compaction-vs-append locking, exhausted-retry handling, and reclaim-guard identity. Residual fs-lock scheduling windows are documented and accepted.
|
||
- Outcome: accepted. F12 passes=true came from the user's own host verification (2,836 null rows backfilled, column default set) and was committed unchanged. Extension repo still owns no DB code for F12.
|
||
- Learning: journal locking is the hard part, not the retry logic — plan the lock protocol before writing the journal. Browser + live-cluster checks remain the only gaps; deploy requires a human.
|