{ "$schema": [ "/home/rawman/.config/opencode/schemas/feature_list.schema.json" ], "features": [ { "feature_id": "F01", "title": "Bundle scaffold (module + hook + endpoint), build pipeline, and local load", "acceptance_steps": [ "Run `npm install` → resolves the pinned devDependencies including vue.", "Run `npm run build` → emits `dist/app.js` and `dist/api.js`; `npm run typecheck` reports 0 errors.", "`directus-extension link` (or the dev compose stack) loads the bundle into a local Directus 11.17.4 with no error.", "The `image-previews` module appears in the admin sidebar, `GET /image-previews/status` answers, and a test upload emits the hook debug line.", "`npm run publish` copies `dist/` + `package.json` to the configured deploy target (staging only at this stage)." ], "passes": false, "notes": [ "Manifest is a bundle: `path:{app,api}`, `entries[{type:module|hook|endpoint,name,source}]`, `host:^10.10.0 || ^11.0.0`.", "Module sidebar fields (id/name/icon/routes) live in the entrypoint object, not package.json." ] }, { "feature_id": "F02", "title": "Staging environment harness + deploy/rollback scripts (safe testing)", "acceptance_steps": [ "`dev/compose.dev.yml` brings up Directus 11.17.4 with its own postgres + redis on a dedicated port, with the extension mounted.", "The stack seeds from a **restored anonymized DB dump** and a **copy of uploads** (`dev/seed/`), so previews can be tested against realistic data.", "A documented script builds and deploys to the staging instance, and a rollback script restores the previous `dist/` and disables the extension.", "Confirmed the local/staging stack shares NOTHING with prod (separate DB, separate storage, separate ports).", "README documents how to start/stop/seed the stack and the exact anonymization rules used for the dump." ], "passes": false, "blocked_by": [ "F01" ], "notes": [ "This must land before any feature is tested on real-ish data; never test on the prod instance.", "Decide staging placement (second compose project on the production host vs a separate dev box) — goal.md open item #5.", "Dev harness (dev/compose.dev.yml + sync-extension.sh) verified; staging compose/deploy/rollback templates added but not executed remotely." ] }, { "feature_id": "F03", "title": "Persisted settings + progress-state store and the config/status API endpoints", "acceptance_steps": [ "`src/lib/settings.ts` reads/writes a JSON settings file under `IMAGE_PREVIEWS_STATE_DIR` (default `/directus/extensions/.image-previews`), merged over env defaults; keys: `enabled` (default false), `windowStart`, `windowEnd`, `concurrency`, `presets`, `formats` (`avif,webp`), `onUpload`, `debug`.", "`GET /image-previews/config` returns effective settings; `PUT /image-previews/config` validates and persists them; invalid payloads return 400 without corrupting the file.", "`src/lib/state.ts` appends completed `{file_id,preset_key,format}` records to `state.jsonl` and loads them back; a resume helper returns completed pairs + totals.", "`GET /image-previews/status` returns `{ running, total, done, remaining, generated, hits, failed, rate, eta, lastRun, nextWindow }`.", "Unit tests cover settings merge/defaults/validation, config round-trip, state append/resume, status aggregation.", "`npm test` green and `npm run typecheck` 0 errors." ], "passes": true, "blocked_by": [ "F02" ], "notes": [ "No native Directus extension-settings store exists; a JSON file under a persisted writable dir is the pragmatic choice.", "All state is file-backed (PM2 cluster, 8 workers — never in-memory)." ] }, { "feature_id": "F04", "title": "Preset registry (system + settings, dynamic) and the in-process warmer core", "acceptance_steps": [ "`src/lib/presets.ts` returns the merged preset list in controller order (`[...SYSTEM_ASSET_ALLOW_LIST, ...storage_asset_presets]`) with first-match-wins on duplicate keys.", "System preset objects come from an `@directus/api` constant import if available, else the version-pinned copy in `src/lib/system-presets.ts`; the mechanism used is recorded in progress.txt.", "`src/lib/warmer.ts` calls `new services.AssetsService({ schema }).getAsset(fileId, { transformationParams: , acceptFormat })` then destroys/unpipes the returned stream.", "For a `format:'auto'` preset the warmer emits BOTH `avif` and `webp`; a fixed-`format` preset emits one; a resize-only preset emits none.", "Concurrency honours the configured limit; a per-preset/format failure (concurrency error, `IllegalAssetTransformation`, missing dims) is caught, logged, and doesn't abort the rest; the result summarises `{ requested, ok, failed }`.", "Unit tests assert: merged preset order + first-match-wins; variant selection per format; exact `getAsset` args; concurrency cap; error isolation.", "`npm test` green and `npm run typecheck` 0 errors." ], "passes": true, "blocked_by": [ "F03" ], "notes": [ "Passing `{ key }` alone is a no-op — always pass the full preset object.", "`acceptFormat:'avif'` in a direct call equals a browser `Accept: image/avif` (verified) — reproduces the admin cache file without HTTP." ] }, { "feature_id": "F05", "title": "Upload hook + new-preset trigger", "acceptance_steps": [ "On `files.upload` where `payload.type` starts with `image/` and `onUpload` is enabled, the hook runs the warmer for configured presets off the response path (fire-and-forget; never blocks or fails the upload).", "A non-image upload performs no warming; a warmer failure never fails or slows the upload.", "After uploading a test JPEG through the admin, the expected `__.avif` (and `.webp`) files for each configured `format:'auto'` preset appear in the storage root.", "When a storage asset preset is added/changed, existing images are warmed for it (`settings.update` hook and/or a manual \"sync presets\" action); re-running is idempotent.", "Under PM2 cluster the hook fires once per upload (logged once, not per worker)." ], "passes": true, "blocked_by": [ "F04" ], "notes": [ "`files.upload` is the correct/only post-write event in v11.17.4.", "New-preset trigger mechanics are goal.md open item #3.", "Decision: manual sync chosen — POST /run is idempotent and warms existing images for all presets; no settings.update auto-trigger." ] }, { "feature_id": "F06", "title": "Diagnostics & manual generation API + observability", "acceptance_steps": [ "`POST /image-previews/generate` `{ fileId, keys?, formats?, dryRun? }` generates (or dry-runs) previews for ONE file and returns per `(key,format)` `{ ok, cached, filename, bytes, ms, error }`.", "`GET /image-previews/resolve-plan?fileId=&key=` returns the resolved transforms array, the **predicted cache filename**, and whether it already exists — WITHOUT generating.", "`GET /image-previews/ops?limit=` returns a recent-operations ring buffer (timestamp, fileId, key, format, ok, ms, error), capped in size.", "A `debug` setting toggles verbose structured logs; logs carry ids/metadata only, never file contents.", "All diagnostics endpoints except `/generate` are read-only and safe to call on a live instance.", "Unit tests cover resolve-plan prediction (given a preset, the predicted filename equals the warmer's produced filename) and ops-ring-buffer behaviour.", "`npm test` green and `npm run typecheck` 0 errors." ], "passes": true, "blocked_by": [ "F04" ], "notes": [ "resolve-plan is the debugging tool that proves hash-match before anything is written.", "Also the autonomous loop's primary feedback channel — keep responses machine-readable." ] }, { "feature_id": "F07", "title": "Admin module UI: progress bar, disk usage, config form, diagnostics panel", "acceptance_steps": [ "The \"Image Previews\" module renders and shows a progress bar with `done / total`, remaining, throughput, ETA, plus last-run and next-window times.", "During a run the page live-updates (polls `/status`); \"Run now\" / \"Cancel\" controls start/stop a run.", "A disk-usage panel shows total bytes + count of preview files (list the storage root, sum `stat()` sizes of `__` files); a refresh control recomputes it.", "A config form edits `enabled`, maintenance window, concurrency, preset selection, formats, `onUpload`, `debug`, persisting via `PUT /config`; changes take effect without rebuilding.", "A diagnostics panel lists recent operations (`/ops`) and lets an operator run generate/dry-run/resolve-plan for a pasted file id.", "Invalid input is rejected client-side with a clear message; a saved change is reflected after reload.", "Smoke-verified on the staging stack (notes/screenshots in progress.txt)." ], "passes": false, "blocked_by": [ "F06" ], "notes": [ "Use Directus UI components / the private-view shell; keep it minimal.", "Concurrency is the throttle knob here because in-process generation bypasses the HTTP rate limiter." ] }, { "feature_id": "F08", "title": "Resumable backfill engine (window-aware, throttled, progress-reporting)", "acceptance_steps": [ "The job enumerates `directus_files` with `type` starting `image/` (paginated) and runs the warmer for each file × configured preset × format.", "It only runs inside the configured maintenance window, or on an explicit manual run; starting outside the window when scheduled is a logged no-op.", "It honours the configured concurrency cap and appends completed pairs to `state.jsonl`; restarting after a kill resumes from the last completed pair.", "Progress is observable during a run via `GET /status` and periodic log lines.", "A second full run over an already-warmed library reports zero new generations (idempotent).", "A >6000px original or a concurrency error is recorded as `failed` for that pair; the job continues.", "`POST /run` starts a run, `POST /run/cancel` stops it; a second concurrent run returns 409 (cluster-wide lock file)." ], "passes": true, "blocked_by": [ "F07" ], "notes": [ "Run state is file-backed; only one run at a time across the cluster.", "Scheduling: external cron hitting `/run` (default) vs in-process timer — goal.md open item #4." ] }, { "feature_id": "F09", "title": "Verification harness: hash-match gate + cold/warm timing + disk counts", "acceptance_steps": [ "For one real file, the on-disk filename produced by an admin browser request for `?key=system-medium-cover` is byte-identical to the warmer's non-dry-run output for the same key+format (the avif variant matches the browser's avif request).", "The same is demonstrated for `system-large-cover` (image field) and `system-small-cover` (image display).", "A timing check prints cold vs warm `GET /assets/?key=system-medium-cover`; warm is materially faster.", "Preview-file count and total bytes are reported before/after a backfill chunk and increase by the expected number of (file × preset × format) entries.", "Findings (observed admin query string, resulting filenames, uuids, timings) are recorded in progress.txt." ], "passes": true, "blocked_by": [ "F08" ], "notes": [ "Make-or-break gate: if the warmer's filename does not match the admin's, the approach is wrong — revisit before rollout.", "Also validates the system-preset copy from F04." ] }, { "feature_id": "F10", "title": "Ops: fail-open/disabled-by-default, prod rollout + rollback, scheduling, docs, remote-feedback runbook", "acceptance_steps": [ "Every entrypoint body is wrapped so a thrown error is logged and swallowed (never propagates into Directus core), and ALL behavior is gated by `enabled` (default false) — a loaded-but-disabled extension is inert, proven by loading it on staging with `enabled=false`.", "A documented prod rollout procedure exists: backup previous `dist/` → deploy → healthcheck → enable via config; and a rollback that restores `dist/` and disables.", "A documented scheduling setup runs the backfill inside the maintenance window (cron hitting `POST /image-previews/run`, by default).", "`README.md` documents purpose, install/build/publish, every setting, presets/formats warmed, run/resume/cancel, disk reporting, and the maintenance window; `AGENTS.md` captures the transform invariants and safety rules.", "A **remote-feedback runbook** documents the scoped token and/or restricted SSH access for the autonomous loop (staging only, no prod Docker socket, no prod admin token, private network), and the machine-readable endpoints used as the feedback channel.", "Final end-to-end pass on staging: upload a fresh image → previews appear with no admin-visible delay; a small backfill chunk completes with the progress bar advancing and disk usage updating; then a controlled prod enable with rollback verified." ], "passes": false, "blocked_by": [ "F09" ], "notes": [ "Promotion to prod is a human action; the loop proposes, a human deploys.", "Record the exact cron entry, the scoped grants, and the rollback commands verbatim." ] }, { "feature_id": "F11", "title": "Persistent failure journal, bounded retry, failure API and module view", "acceptance_steps": [ "Every failed (file, preset, format) task from the upload hook, manual `/generate`, and backfill appends an event to `/failures.jsonl`; replay derives the latest state per key across restarts and PM2 workers.", "Default retry policy is max 3 attempts (initial + 2 retries) with 5 min then 15 min backoff; a failed task gets at least one automatic retry when the scheduler next runs with no active backfill inside the maintenance window.", "Due retries run under the existing `run.lock`, process only the targeted file/preset/format pairs (bounded batch, default 50), and never trigger a full-library sweep; a manual `Run now` may ignore the window.", "A recovered task appends a normal `state.jsonl` completion and is marked `recovered`; an exhausted task stays `failed` with attempts, last error, and timestamps, and is not retried again.", "`GET /image-previews/failures?status=&limit=&offset=` returns persisted records plus file metadata (`filename_download`, `type`, `folder` when available), attempts, first/last failure, next retry, status, and error text; admin-only and read-only.", "The admin module shows a \"Failed previews\" table with file, preset, format, attempts, last error, next retry, and status; no toasts or user-facing notifications are added.", "Every failure logs a structured warn with `fileId`, `preset_key`, `format`, `attempt`, `next_retry_at`, and `error`; recovery logs at info. No file contents are logged.", "Journal compaction under the run lock retains unresolved rows plus the last 1,000 recovered rows, so the file cannot grow without bound.", "Targeted unit tests cover journal append/replay/idempotency, backoff schedule, max-attempt exhaustion, due-retry selection, and the failure endpoint response shape." ], "passes": true, "blocked_by": [ "F08" ], "notes": [ "Root cause: `/ops` is an in-memory 200-entry ring buffer per PM2 worker; after a long run the 17 failure details were unrecoverable.", "Do not reuse `/ops` for persistence; keep it as the live view. Journal is append-only for cluster safety; compact only under `run.lock`.", "Retry policy starts with constants; expose settings only if tuning is needed after the first real run.", "No user-facing notifications; logs and the read-only API/module table are the observability surface.", "Accepted 2026-10-30 on code + test evidence (79/79, tsc clean, build OK) after three review rounds; browser module render and live PM2 scheduler/retry tick were waived by the user and remain unverified." ] }, { "feature_id": "F12", "title": "Host fix: `uploaded_on` integrity for TUS and legacy files (no Directus upgrade)", "acceptance_steps": [ "Before change, record `count(*)`, `count(*) filter (where uploaded_on is null)`, and `count(*) filter (where created_on is null)` for `directus_files`; confirm the legacy null block explains the file-library ordering.", "In a transaction after a table backup, run `update directus_files set uploaded_on = coalesce(created_on, now()) where uploaded_on is null;` and verify only `uploaded_on` changed (`modified_on`/`modified_by` unchanged, no new activity/revisions).", "Set the column default to `(now() at time zone 'utc')` when absent, so future TUS inserts that omit `uploaded_on` receive a date.", "Upload one test file with `TUS_ENABLED=true`; confirm `uploaded_on` is non-null, `tus_id`/`tus_data` are null after finish, and the file appears at the top of the unfiltered folder with `-uploaded_on`.", "Keep `TUS_ENABLED=true`; no Directus upgrade and no extension code change required for this fix.", "Document latent 11.17.4 TUS defects not changed here: client-declared `type`/`filesize`, and stale `tus_id` placeholders hidden from `/files` if finish/cleanup fails." ], "passes": true, "blocked_by": [], "notes": [ "Source: v11.17.4 `api/src/services/tus/{data-store,server}.ts` omits `uploaded_on`; only `/files` (`FilesService.uploadOne`) sets it. Upstream #28042 fixed in 12.4.0, not backported to 11.x.", "Raw SQL is the only way to avoid bumping `modified_on`/`modified_by`; `ItemsService.updateOne` always runs payload specials even with `emitEvents:false`.", "TUS row insert omits `uploaded_on` entirely, so a DB default applies to future inserts. Host `TimeZone=Etc/UTC`; `(now() at time zone 'utc')::timestamptz` matches Directus's UTC writes.", "Verified 2026-10-30: 2,836 null rows backfilled from `created_on`; 39,350/39,350 dated; latest 5 TUS rows have non-null `uploaded_on`, `tus_id=null`, delta vs `created_on` 0–0.24s; zero `idle in transaction`.", "Apparent inverted sort is a UI icon issue, not data: Directus cards `header.vue` flips the sort glyph for descending, opposite to tabular. Network `sort[]=-uploaded_on` = newest first; `sort[]=uploaded_on` = oldest first. No server/data fix needed.", "This is host data remediation, not extension code; keep it separate from the preview warmer." ] } ] }