docs: add failure journal + uploaded_on features to goal artifacts

.opencode/goal.md

@@ -3,6 +3,7 @@
 Status: SCAFFOLDED — opened 2026-10-25. Standalone extension project (independent of any host
 application).
 Location: `/home/rawman/Projects/DirectusExtensions/directus-extension-generate-image-previews`
+Branch: `goal/image-previews` (created from master 2026-10-30).
 Target: self-hosted Directus **11.17.4** (PM2 cluster, local disk storage).
 Project type: Directus **bundle** extension (app module + api hook + api endpoint).
 
@@ -112,6 +113,48 @@ Our registry (`src/lib/presets.ts`) mirrors this exact list:
 - `debug` setting enables verbose structured logs; logs carry ids/metadata only, never file contents.
 - Everything except `/generate` is read-only.
 
+## Failure journal & bounded retries (2026-10-30)
+
+Backfill failures are currently only counted in `run.json` and held in the in-memory `/ops` ring
+buffer (last 200 entries, per PM2 worker). After a long run the details are lost. Add a durable
+failure journal and a bounded retry path.
+
+- `failures.jsonl` under the state dir = append-only events `{type:'failed'|'retry'|'recovered',
+  ts, file_id, preset_key, format, attempt, next_retry_at?, error?, ms?}`. Replay derives the latest
+  state per `(file_id, preset_key, format)`; append-only so PM2 workers can write safely.
+- Every failure (upload hook, manual `/generate`, backfill) is journaled. Default retry policy:
+  max 3 attempts (initial + 2 retries), backoff 5 min then 15 min. Due retries run only while enabled,
+  inside the maintenance window (manual `Run now` may ignore it), with no active backfill, under the
+  same `run.lock`; targeted file/preset/format retries — never a full-library sweep.
+- Success after retry appends a normal `state.jsonl` completion and marks the failure `recovered`.
+  Exhausted retries stay `failed` with the last error and remain readable.
+- `GET /image-previews/failures?status=&limit=&offset=` returns persisted records + file metadata,
+  attempts, timestamps, next retry and error text. Admin-only, read-only.
+- Module: "Failed previews" table (file, preset, format, attempts, last error, next retry, status).
+  No toasts or user-facing notifications.
+- Log every failure at warn with ids/metadata + error; log recoveries at info. No file contents.
+- Journal is compacted under the run lock: keep unresolved rows + last 1,000 recovered; never grow
+  unbounded. `/ops` remains the live in-memory view.
+
+## Host data fix: `uploaded_on` for TUS uploads (2026-10-30)
+
+Scope split (user decision): DB access and execution stay with the user; this repo only ships the SQL
+script + runbook. No extension code change for this item.
+
+Directus 11.17.4 TUS uploads omit `uploaded_on` (`TusDataStore.create` + `onUploadFinish`); the
+regular `POST /files` path sets it. Upstream issue #28042 is fixed only in 12.4.0; this host stays on
+11.17.4.
+
+- Fix existing rows directly in Postgres: `update directus_files set uploaded_on =
+  coalesce(created_on, now()) where uploaded_on is null;` in a transaction, after a table backup.
+  Raw SQL touches only `uploaded_on`; it does not bump `modified_on`/`modified_by`, emit events, or
+  create revisions.
+- Add a UTC column default so future TUS inserts get a date:
+  `alter table directus_files alter column uploaded_on set default (now() at time zone 'utc');`
+  Confirm one new TUS upload has non-null `uploaded_on` and `tus_id = null`.
+- Keep `TUS_ENABLED=true`; no Directus upgrade and no extension code required. Latent TUS defects
+  (client-declared `type`/`filesize`, stale `tus_id` placeholders) are documented, not changed.
+
 ## Safe deployment & staging (must not crash prod)
 
 - **Fail-open, inert by default**: every entrypoint is tiny, bodies wrapped in try/catch, hooks never
@@ -157,6 +200,8 @@ src/
     warmer.ts            in-process getAsset(x, { transformationParams, acceptFormat }) + concurrency
     backfill.ts          enumerate directus_files (image/*), window-aware, resumable
     state.ts             state.jsonl read/append/resume
+    failures.ts          failures.jsonl append/replay/compact
+    retry.ts             due-retry selection + backoff/max-attempt policy
     settings.ts          config file read/write + env defaults
     disk.ts              sum sizes of __<hash> files (driver.list + stat)
     window.ts            maintenance-window logic (is-now-in-window, next-window)