Rawman/directus-extension-generate-image-previews
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chore: prepare for public release

Browse source 
- add repository/homepage/bugs metadata; target Directus ^11
- genericize internal deployment references
- document install, compatibility, scheduling, security
- stop tracking internal .opencode work artifacts
This commit is contained in:
Rawman Grey 2026-10-25 12:25:19 +03:00
parent c8f8bbc5ac
commit 94f197bdec
10 changed files with 181 additions and 706 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -3,3 +3,4 @@ node_modules
dist dist
dev/extension dev/extension
dev/.smoke.env dev/.smoke.env
.opencode/

1
.opencode/.gitignore vendored
View file

@ -1 +0,0 @@
node_modules

206
.opencode/feature_list.json
View file

@ -1,206 +0,0 @@
{
"$schema": [
"/home/rawman/.config/opencode/schemas/feature_list.schema.json"
],
"features": [
{
"feature_id": "F01",
"title": "Bundle scaffold (module + hook + endpoint), build pipeline, and local load",
"acceptance_steps": [
"Run `npm install` → resolves the pinned devDependencies including vue.",
"Run `npm run build` → emits `dist/app.js` and `dist/api.js`; `npm run typecheck` reports 0 errors.",
"`directus-extension link` (or the dev compose stack) loads the bundle into a local Directus 11.17.4 with no error.",
"The `image-previews` module appears in the admin sidebar, `GET /image-previews/status` answers, and a test upload emits the hook debug line.",
"`npm run publish` copies `dist/` + `package.json` to the configured deploy target (staging only at this stage)."
],
"passes": false,
"notes": [
"Manifest is a bundle: `path:{app,api}`, `entries[{type:module|hook|endpoint,name,source}]`, `host:^10.10.0 || ^11.0.0`.",
"Module sidebar fields (id/name/icon/routes) live in the entrypoint object, not package.json."
]
},
{
"feature_id": "F02",
"title": "Staging environment harness + deploy/rollback scripts (safe testing)",
"acceptance_steps": [
"`dev/compose.dev.yml` brings up Directus 11.17.4 with its own postgres + redis on a dedicated port, with the extension mounted.",
"The stack seeds from a **restored anonymized DB dump** and a **copy of uploads** (`dev/seed/`), so previews can be tested against realistic data.",
"A documented script builds and deploys to the staging instance, and a rollback script restores the previous `dist/` and disables the extension.",
"Confirmed the local/staging stack shares NOTHING with prod (separate DB, separate storage, separate ports).",
"README documents how to start/stop/seed the stack and the exact anonymization rules used for the dump."
],
"passes": false,
"blocked_by": [
"F01"
],
"notes": [
"This must land before any feature is tested on real-ish data; never test on the prod instance.",
"Decide staging placement (second compose project on land-nest vs a separate dev box) — goal.md open item #5.",
"Dev harness (dev/compose.dev.yml + sync-extension.sh) verified; staging compose/deploy/rollback templates added but not executed remotely."
]
},
{
"feature_id": "F03",
"title": "Persisted settings + progress-state store and the config/status API endpoints",
"acceptance_steps": [
"`src/lib/settings.ts` reads/writes a JSON settings file under `IMAGE_PREVIEWS_STATE_DIR` (default `/directus/extensions/.image-previews`), merged over env defaults; keys: `enabled` (default false), `windowStart`, `windowEnd`, `concurrency`, `presets`, `formats` (`avif,webp`), `onUpload`, `debug`.",
"`GET /image-previews/config` returns effective settings; `PUT /image-previews/config` validates and persists them; invalid payloads return 400 without corrupting the file.",
"`src/lib/state.ts` appends completed `{file_id,preset_key,format}` records to `state.jsonl` and loads them back; a resume helper returns completed pairs + totals.",
"`GET /image-previews/status` returns `{ running, total, done, remaining, generated, hits, failed, rate, eta, lastRun, nextWindow }`.",
"Unit tests cover settings merge/defaults/validation, config round-trip, state append/resume, status aggregation.",
"`npm test` green and `npm run typecheck` 0 errors."
],
"passes": true,
"blocked_by": [
"F02"
],
"notes": [
"No native Directus extension-settings store exists; a JSON file under a persisted writable dir is the pragmatic choice.",
"All state is file-backed (PM2 cluster, 8 workers — never in-memory)."
]
},
{
"feature_id": "F04",
"title": "Preset registry (system + settings, dynamic) and the in-process warmer core",
"acceptance_steps": [
"`src/lib/presets.ts` returns the merged preset list in controller order (`[...SYSTEM_ASSET_ALLOW_LIST, ...storage_asset_presets]`) with first-match-wins on duplicate keys.",
"System preset objects come from an `@directus/api` constant import if available, else the version-pinned copy in `src/lib/system-presets.ts`; the mechanism used is recorded in progress.txt.",
"`src/lib/warmer.ts` calls `new services.AssetsService({ schema }).getAsset(fileId, { transformationParams: <verbatim preset object>, acceptFormat })` then destroys/unpipes the returned stream.",
"For a `format:'auto'` preset the warmer emits BOTH `avif` and `webp`; a fixed-`format` preset emits one; a resize-only preset emits none.",
"Concurrency honours the configured limit; a per-preset/format failure (concurrency error, `IllegalAssetTransformation`, missing dims) is caught, logged, and doesn't abort the rest; the result summarises `{ requested, ok, failed }`.",
"Unit tests assert: merged preset order + first-match-wins; variant selection per format; exact `getAsset` args; concurrency cap; error isolation.",
"`npm test` green and `npm run typecheck` 0 errors."
],
"passes": true,
"blocked_by": [
"F03"
],
"notes": [
"Passing `{ key }` alone is a no-op — always pass the full preset object.",
"`acceptFormat:'avif'` in a direct call equals a browser `Accept: image/avif` (verified) — reproduces the admin cache file without HTTP."
]
},
{
"feature_id": "F05",
"title": "Upload hook + new-preset trigger",
"acceptance_steps": [
"On `files.upload` where `payload.type` starts with `image/` and `onUpload` is enabled, the hook runs the warmer for configured presets off the response path (fire-and-forget; never blocks or fails the upload).",
"A non-image upload performs no warming; a warmer failure never fails or slows the upload.",
"After uploading a test JPEG through the admin, the expected `<basename>__<hash>.avif` (and `.webp`) files for each configured `format:'auto'` preset appear in the storage root.",
"When a storage asset preset is added/changed, existing images are warmed for it (`settings.update` hook and/or a manual \"sync presets\" action); re-running is idempotent.",
"Under PM2 cluster the hook fires once per upload (logged once, not per worker)."
],
"passes": true,
"blocked_by": [
"F04"
],
"notes": [
"`files.upload` is the correct/only post-write event in v11.17.4.",
"New-preset trigger mechanics are goal.md open item #3.",
"Decision: manual sync chosen — POST /run is idempotent and warms existing images for all presets; no settings.update auto-trigger."
]
},
{
"feature_id": "F06",
"title": "Diagnostics & manual generation API + observability",
"acceptance_steps": [
"`POST /image-previews/generate` `{ fileId, keys?, formats?, dryRun? }` generates (or dry-runs) previews for ONE file and returns per `(key,format)` `{ ok, cached, filename, bytes, ms, error }`.",
"`GET /image-previews/resolve-plan?fileId=&key=` returns the resolved transforms array, the **predicted cache filename**, and whether it already exists — WITHOUT generating.",
"`GET /image-previews/ops?limit=` returns a recent-operations ring buffer (timestamp, fileId, key, format, ok, ms, error), capped in size.",
"A `debug` setting toggles verbose structured logs; logs carry ids/metadata only, never file contents.",
"All diagnostics endpoints except `/generate` are read-only and safe to call on a live instance.",
"Unit tests cover resolve-plan prediction (given a preset, the predicted filename equals the warmer's produced filename) and ops-ring-buffer behaviour.",
"`npm test` green and `npm run typecheck` 0 errors."
],
"passes": true,
"blocked_by": [
"F04"
],
"notes": [
"resolve-plan is the debugging tool that proves hash-match before anything is written.",
"Also the autonomous loop's primary feedback channel — keep responses machine-readable."
]
},
{
"feature_id": "F07",
"title": "Admin module UI: progress bar, disk usage, config form, diagnostics panel",
"acceptance_steps": [
"The \"Image Previews\" module renders and shows a progress bar with `done / total`, remaining, throughput, ETA, plus last-run and next-window times.",
"During a run the page live-updates (polls `/status`); \"Run now\" / \"Cancel\" controls start/stop a run.",
"A disk-usage panel shows total bytes + count of preview files (list the storage root, sum `stat()` sizes of `__<hash>` files); a refresh control recomputes it.",
"A config form edits `enabled`, maintenance window, concurrency, preset selection, formats, `onUpload`, `debug`, persisting via `PUT /config`; changes take effect without rebuilding.",
"A diagnostics panel lists recent operations (`/ops`) and lets an operator run generate/dry-run/resolve-plan for a pasted file id.",
"Invalid input is rejected client-side with a clear message; a saved change is reflected after reload.",
"Smoke-verified on the staging stack (notes/screenshots in progress.txt)."
],
"passes": false,
"blocked_by": [
"F06"
],
"notes": [
"Use Directus UI components / the private-view shell; keep it minimal.",
"Concurrency is the throttle knob here because in-process generation bypasses the HTTP rate limiter."
]
},
{
"feature_id": "F08",
"title": "Resumable backfill engine (window-aware, throttled, progress-reporting)",
"acceptance_steps": [
"The job enumerates `directus_files` with `type` starting `image/` (paginated) and runs the warmer for each file × configured preset × format.",
"It only runs inside the configured maintenance window, or on an explicit manual run; starting outside the window when scheduled is a logged no-op.",
"It honours the configured concurrency cap and appends completed pairs to `state.jsonl`; restarting after a kill resumes from the last completed pair.",
"Progress is observable during a run via `GET /status` and periodic log lines.",
"A second full run over an already-warmed library reports zero new generations (idempotent).",
"A >6000px original or a concurrency error is recorded as `failed` for that pair; the job continues.",
"`POST /run` starts a run, `POST /run/cancel` stops it; a second concurrent run returns 409 (cluster-wide lock file)."
],
"passes": true,
"blocked_by": [
"F07"
],
"notes": [
"Run state is file-backed; only one run at a time across the cluster.",
"Scheduling: external cron hitting `/run` (default) vs in-process timer — goal.md open item #4."
]
},
{
"feature_id": "F09",
"title": "Verification harness: hash-match gate + cold/warm timing + disk counts",
"acceptance_steps": [
"For one real file, the on-disk filename produced by an admin browser request for `?key=system-medium-cover` is byte-identical to the warmer's non-dry-run output for the same key+format (the avif variant matches the browser's avif request).",
"The same is demonstrated for `system-large-cover` (image field) and `system-small-cover` (image display).",
"A timing check prints cold vs warm `GET /assets/<id>?key=system-medium-cover`; warm is materially faster.",
"Preview-file count and total bytes are reported before/after a backfill chunk and increase by the expected number of (file × preset × format) entries.",
"Findings (observed admin query string, resulting filenames, uuids, timings) are recorded in progress.txt."
],
"passes": true,
"blocked_by": [
"F08"
],
"notes": [
"Make-or-break gate: if the warmer's filename does not match the admin's, the approach is wrong — revisit before rollout.",
"Also validates the system-preset copy from F04."
]
},
{
"feature_id": "F10",
"title": "Ops: fail-open/disabled-by-default, prod rollout + rollback, scheduling, docs, remote-feedback runbook",
"acceptance_steps": [
"Every entrypoint body is wrapped so a thrown error is logged and swallowed (never propagates into Directus core), and ALL behavior is gated by `enabled` (default false) — a loaded-but-disabled extension is inert, proven by loading it on staging with `enabled=false`.",
"A documented prod rollout procedure exists: backup previous `dist/` → deploy → healthcheck → enable via config; and a rollback that restores `dist/` and disables.",
"A documented scheduling setup runs the backfill inside the maintenance window (cron hitting `POST /image-previews/run`, by default).",
"`README.md` documents purpose, install/build/publish, every setting, presets/formats warmed, run/resume/cancel, disk reporting, and the maintenance window; `AGENTS.md` captures the transform invariants and safety rules.",
"A **remote-feedback runbook** documents the scoped token and/or restricted SSH access for the autonomous loop (staging only, no prod Docker socket, no prod admin token, private network), and the machine-readable endpoints used as the feedback channel.",
"Final end-to-end pass on staging: upload a fresh image → previews appear with no admin-visible delay; a small backfill chunk completes with the progress bar advancing and disk usage updating; then a controlled prod enable with rollback verified."
],
"passes": false,
"blocked_by": [
"F09"
],
"notes": [
"Promotion to prod is a human action; the loop proposes, a human deploys.",
"Record the exact cron entry, the scoped grants, and the rollback commands verbatim."
]
}
]
}

249
.opencode/goal.md
View file

@ -1,249 +0,0 @@
# Goal: Directus image preview pre-generation
Status: SCAFFOLDED — opened 2026-10-25. Standalone extension project (not part of the
landshaft.life app repo).
Location: `/home/rawman/Projects/DirectusExtensions/directus-extension-generate-image-previews`
Target server: `land-nest`, Directus **11.17.4** (self-hosted, PM2 cluster, local disk storage).
Project type: Directus **bundle** extension (app module + api hook + api endpoint).
## Summary
Pre-generate Directus image transform ("preview") cache files so the admin UI never pays the
first-request sharp cost. One warmer, several triggers:
- **upload hook** — after an image upload, warm its previews.
- **new-preset trigger** — when a storage asset preset is added/changed, warm it for existing images.
- **backfill job** — a resumable, scheduled sweep over the existing `directus_files` library, inside a
configurable maintenance window.
- **manual / diagnostics** — generate (or dry-run) previews for a single file on demand, to debug.
Plus an **admin module** ("Image Previews") with a **progress bar**, **disk usage**, a **config form**
(persisted, no rebuild), and a **diagnostics panel**.
## Verified mechanism (source-checked against tag v11.17.4)
The on-disk transform cache file is:
```
<basename(filename_disk)>__<object-hash(resolved transforms)>.<ext>
```
written **beside the original** in the storage root (`STORAGE_LOCAL_ROOT=./uploads`).
- `hash` = `object-hash@3` defaults: sha1/hex, `unorderedArrays:false` (**array order matters**),
`unorderedObjects:true`. Hashed over the **RESOLVED** transform array from `resolvePreset`.
- `resolvePreset` order is fixed: `(1) copy transformationParams.transforms[]`, `(2) push toFormat`
(if `format`/`quality`), `(3) push resize` (+`extract` on focal cover).
- `maybeExtractFormat` picks the last `toFormat` arg as the file extension.
- `AssetsService.getAsset` checks `exists(assetFilename)` first (cache hit → no sharp); on miss runs
sharp and **awaits** `storage.write(...)` before returning. Not draining the returned stream only
leaks an FD (never truncates).
- Called as sudo (`new AssetsService({ schema })`) it needs **no** HTTP request context.
## Why the naive / community approach FAILS (must not repeat)
1. **`{ key }` alone is a no-op.** `resolvePreset` never reads `key`. Pass the **full preset object**.
2. **`format:'auto'` mismatch.** Every `system-*` preset is `format:'auto'`. `getFormat` with no
`acceptFormat` → JPEG→`jpg`/PNG→`png`; a browser `Accept: image/avif` → `avif`. Different
`toFormat` arg → different hash → **guaranteed miss**.
3. Disproved myths: the write is awaited; the admin does NOT use `width`/`height`.
4. Raw params resolve to `[toFormat, resize]`; a preset resolves to `[resize, toFormat]` → different hash.
## Chosen strategy — IN-PROCESS generation (no token, no HTTP, no rate limiter)
Because this is an internal extension, generate by calling Directus's own service in-process:
```ts
const assets = new services.AssetsService({ schema }); // sudo
const { stream } = await assets.getAsset(fileId, {
transformationParams: presetObject, // FULL object, not { key }
acceptFormat: 'avif', // or 'webp'
});
(stream as Readable).destroy?.();
```
`acceptFormat` reproduces exactly what a browser's `Accept` header does, so the cache filename is
byte-identical to the admin/browser read — **without any HTTP request, token, proxy, or rate limiter**.
Generate **two variants** for `format:'auto'` presets (`avif`, `webp`); fixed-`format` presets yield
one; resize-only presets (no `format`/`quality`) yield none.
HTTP-replay (`GET /assets/{id}?key=…` with a browser `Accept` + auth) is the documented **fallback**
only — it needs a token and rate-limiter pacing, which is why it is not the default.
## Preset registry (system + settings, dynamic)
The controller builds its preset list inline as:
```
res.locals['shortcuts'] = [...SYSTEM_ASSET_ALLOW_LIST, ...(directus_settings.storage_asset_presets ?? [])]
```
and resolves `?key=` with `.find(t => t.key === key)` → **first match wins** (a system key shadows a
custom preset of the same key). There is **no** reusable helper and `GET /settings` exposes **only**
custom presets.
Our registry (`src/lib/presets.ts`) mirrors this exact list:
- **System presets**: 6 objects in `SYSTEM_ASSET_ALLOW_LIST` (`system-small|medium|large` ×
`cover|contain`), each `format:'auto'`, resize inside `transforms[]`; sourced at runtime from
`@directus/api` **if** importable, else the version-pinned copy `src/lib/system-presets.ts` (v11.17.4).
- **Custom presets**: read `directus_settings.storage_asset_presets` each run; pass the verbatim object.
- **Dynamic**: newly created presets are picked up per run; the new-preset trigger warms them for
existing images.
## Admin UI (module "Image Previews")
- **Progress**: bar + `done / total / remaining`, throughput, ETA, last-run, next-window; live during a run.
- **Storage breakdown**: bytes + file counts separately for **preview assets** (`__<hash>`), **original
assets**, and the total — raw and human-readable. Computed by listing the storage root and `stat()`-ing
each file (listing yields paths only, no sizes); the state dir is excluded.
- **Config form** (persisted, no rebuild; defaults `enabled:true`, window `01:00–04:00`): `enabled`,
maintenance window, concurrency, preset selection, formats, `onUpload`, `debug`; each field carries a
plain-language explanation. **Run now** ignores the window; **Cancel** is shown only while a run is active.
- **Diagnostics panel**: recent operations, per-file generate/dry-run, resolve-plan viewer.
## Diagnostics & manual generation (observability)
- `POST /image-previews/generate` `{ fileId, keys?, formats?, dryRun? }` → generate (or dry-run) for ONE
file; returns per `(key,format)` `{ ok, failed, cached, filename, bytes, ms, error }`. Primary dev tool.
- `GET /image-previews/resolve-plan?fileId=&key=` → resolved transforms + **predicted cache filename** +
whether it already exists, WITHOUT generating. The hash-match debugging tool.
- `GET /image-previews/ops?limit=` → recent-operations ring buffer (timestamp, fileId, key, format, ok, ms, error).
- `debug` setting enables verbose structured logs; logs carry ids/metadata only, never file contents.
- Everything except `/generate` is read-only.
## Safe deployment & staging (must not crash prod)
- **Fail-open, inert by default**: every entrypoint is tiny, bodies wrapped in try/catch, hooks never
throw into Directus core, and ALL behavior is gated by `enabled` (default `false`). A loaded-but-disabled
extension does nothing.
- **Local dev**: `directus-extension link` (or the dev compose stack) loads `src/` live for iteration.
- **Staging stack**: a second compose project (`dev/compose.dev.yml`) running the SAME Directus 11.17.4
image with its own pg+redis, a **restored anonymized dump**, and a **copy of uploads**, on a separate
port — realistic data, zero prod risk.
- **Rollout discipline**: build → verify on staging → back up the previous `dist/` → copy in → healthcheck
→ enable via config → rollback = restore previous `dist/` + disable.
- `EXTENSIONS_AUTO_RELOAD` in prod reloads on file change; deploy with it controlled and accept the brief
reload, or restart deliberately.
## Remote feedback & autonomous operation (security)
- Feedback channel = the extension's own **machine-readable endpoints** (`/status`, `/ops`,
`/disk-usage`, `/resolve-plan`). Automation reads those, not CMS content.
- **Least privilege**:
- A Directus **admin token = full CMS control**. Do NOT give it to an autonomous loop. Use a scoped
token reaching only `/image-previews/*` (+ `directus_files` metadata read).
- A **Docker socket = root-equivalent**. Never expose the prod socket to automation; keep the loop on staging.
- Deployment access = a dedicated **non-sudo** SSH user with a restricted key (`restrict`, `from=`,
`command=`), or run opencode on the staging host via `handoff`.
- **Network**: staging reachable only over a private network / VPN / tunnel — never public.
- **Prompt-injection**: treat logs, filenames, and image metadata as untrusted input; keep tooling scoped.
- **Promotion to prod is a human action**: the loop proposes, a human merges/deploys.
- Guard long autonomous runs with a cost/time budget.
## Design
```
src/
hook/index.ts (api) files.upload -> warmer; settings/presets change -> warm existing
endpoint/index.ts (api) routes: /config (GET/PUT), /status, /run, /run/cancel,
/disk-usage, /generate, /resolve-plan, /ops
module/
index.ts (app) defineModule({ id:'image-previews', name, icon, routes })
module.vue progress + disk usage + config form + diagnostics panel
lib/
presets.ts merged registry (system + settings), mirrors controller semantics
system-presets.ts version-pinned SYSTEM_ASSET_ALLOW_LIST copy (v11.17.4)
warmer.ts in-process getAsset(x, { transformationParams, acceptFormat }) + concurrency
backfill.ts enumerate directus_files (image/*), window-aware, resumable
state.ts state.jsonl read/append/resume
settings.ts config file read/write + env defaults
disk.ts sum sizes of __<hash> files (driver.list + stat)
window.ts maintenance-window logic (is-now-in-window, next-window)
diagnostics.ts resolve-plan/dry-run (resolved transforms + predicted filename), ops ring buffer
dev/
compose.dev.yml local Directus 11.17.4 + postgres + redis, extension linked
seed/ anonymized DB dump + sample images for local/staging runs
```
## Non-goals
- No Directus upgrade (server stays on 11.17.4; license constraint).
- No change to admin preset configuration or to how the admin builds URLs.
- Never hand-write `__<hash>.<ext>` filenames.
- No CDN/nginx caching work (separate ops concern). No image `quality` policy change.
- Automation never gets prod Docker socket or a full prod admin token.
## Constraints / risks
- `ASSETS_TRANSFORM_MAX_CONCURRENT=45` per worker; over it errors. Catch per file/preset and continue
(never fail the upload).
- `ASSETS_TRANSFORM_IMAGE_MAX_DIMENSION` default 6000 is a **pixel dimension** guard in Directus, not a
file-size limit. The extension imposes no limit of its own; raise the Directus env var to process larger
images (watch memory). Upload-size limits (`FILES_MAX_UPLOAD_SIZE`, `MAX_PAYLOAD_SIZE`) are unrelated.
- Local storage write is **non-atomic** → warm only AFTER an upload is committed; serialize so the
upload hook and the backfill never generate the same file concurrently.
- PM2 cluster (8 workers): the upload hook fires on the handling worker; the backfill runs in the worker
that served the run request. Shared state MUST be file/DB-backed (never in-memory).
- In-process backfill competes with live API serving → the maintenance window + concurrency cap govern
that. (HTTP rate limiter is irrelevant in-process.)
- Observed server bottleneck (2026-10-23): `storage:local:responseTime 1206ms`, `pg:responseTime`
150–738ms; no 503s, no sharp errors.
## Verification policy
- `npm run typecheck` → 0 errors; `npm test` (node:test via tsx) green.
- **The hash-match gate (F09)**: for a real file, the on-disk filename produced by a browser/admin
request for `?key=system-medium-cover` is byte-identical to the warmer's output for the same key+format
(+ the resolved extension matches, e.g. `.avif`).
- Cold-vs-warm timing; preview-file count + total bytes before/after a run.
- All feature testing happens on the **staging** stack; prod deploy only at goal completion, via the
documented backup/rollback procedure.
## Expected files
- `package.json`, `tsconfig.json`, `.gitignore`
- `src/hook/`, `src/endpoint/`, `src/module/`, `src/lib/`, plus `src/**/*.test.ts`
- `dev/compose.dev.yml`, `dev/seed/`
- `.opencode/goal.md`, `.opencode/feature_list.json`, `.opencode/progress.txt`
- `README.md`, `AGENTS.md`
## Resolved decisions (2026-10-25)
1. State/config: JSON files under `<STORAGE_LOCAL_ROOT>/.image-previews` (default
`/directus/uploads/.image-previews`), override with `IMAGE_PREVIEWS_STATE_DIR`. The former
`/directus/extensions/.image-previews` default was rejected — root-owned volume → `EACCES`.
2. System presets: version-pinned copy (`src/lib/system-presets.ts`, verbatim v11.17.4); no
`@directus/api` import needed for presets.
3. New-preset trigger: manual `POST /run` (idempotent) + the module's Run button; no
`settings.update` auto-trigger.
4. Backfill scheduling: external cron → `POST /image-previews/run`.
5. Dev/staging: isolated local compose stack (`dev/compose.dev.yml`). A land-nest staging project is
still to be created for realistic-data runs.
6. Autonomous-loop grants: still to be defined (scoped token / restricted SSH, staging only).
## Implementation status (2026-10-25)
Implemented and verified against an isolated Directus 11.17.4: settings/state, presets + warmer,
upload hook, diagnostics, backfill, endpoints. `npm test` 32/32, typecheck clean, build emits both
bundles. **Hash-match gate passed** (a real browser-style asset request produced exactly the
predicted filename). Features F03–F06, F08, F09 pass.
Outstanding: admin-module browser verification (F07), staging deploy/rollback + prod rollout (F02/F10),
and the F01 publish-to-staging step.
## Reference: source anchors (v11.17.4)
- `api/src/services/assets.ts` — `getAsset`, `getAssetSuffix`, filename build, awaited `write`.
- `api/src/utils/transformations.ts` — `resolvePreset`, `getFormat`.
- `api/src/controllers/assets.ts` — inline `shortcuts = [...SYSTEM_ASSET_ALLOW_LIST, ...settings]`,
`acceptFormat` selection from `req.headers.accept`.
- `api/src/constants.ts` — `SYSTEM_ASSET_ALLOW_LIST` (6 presets, `format:'auto'`);
`ASSETS_TRANSFORM_IMAGE_MAX_DIMENSION` default 6000.
- `app/src/utils/get-asset-url.ts`, `app/src/layouts/cards/components/card.vue`,
`app/src/interfaces/file-image/file-image.vue`, `app/src/displays/image/image.vue` — admin URLs:
Files grid `?key=system-medium-cover&v=<modified_on>`; image field `system-large-cover|contain`;
image display `system-small-cover`.
- `packages/storage-driver-local/src/index.ts` — non-atomic `write`; `list()` returns paths only.
- Prior art that does NOT work as-is: `directus-extension-auto-generate-file-transformations`
(utomic-media) — still falls foul of the `auto`→format mismatch unless it forces the format.

136
.opencode/progress.txt
View file

@ -1,136 +0,0 @@
# progress.txt — directus-extension-generate-image-previews
Append-only log of decisions, evidence, and learnings. Newest at the bottom.
## 2026-10-25 — project scaffolded
- Opened this standalone extension project to pre-generate Directus image previews.
- Source-verified the strategy against Directus **v11.17.4** (see `.opencode/goal.md`).
- Root causes why existing/community extensions do not work:
1. `resolvePreset` never reads `key` → `{transformationParams:{key}}` yields `transforms=[]` → no-op.
2. `system-*` presets are `format:'auto'` → `jpg`/`png` server-side vs `avif` in-browser → different hash.
3. "undrained stream truncates file" — false (write is awaited).
4. admin uses `?key=system-*`, not raw width/height.
- Admin URLs (v11.17.4 app source): files grid `?key=system-medium-cover&v=<modified_on>`;
image field `system-large-cover|contain`; image display `system-small-cover`.
- Server bottleneck evidence (2026-10-23 log triage): `storage:local:responseTime 1206ms`,
`pg:responseTime` 150–738ms; no 503s, no sharp errors.
## 2026-10-25 — scope expanded after user direction (v2)
- **No token needed.** Dropped HTTP-replay as primary. Strategy: **in-process**
`AssetsService.getAsset(id, { transformationParams: <full preset object>, acceptFormat: 'avif'|'webp' })`.
No HTTP → no token, no proxy, no rate limiter. (HTTP-replay kept as documented fallback.)
- **Bundle extension**: api `hook` + api `endpoint` + app `module`.
- **Presets: all, dynamically** — system `SYSTEM_ASSET_ALLOW_LIST` (6) + `directus_settings.storage_asset_presets`,
controller order `[...SYSTEM, ...custom]`, first-match-wins; new presets picked up per run.
- **Admin module**: progress bar, disk usage (`list`+`stat`), config form, diagnostics panel.
- **Config**: maintenance window + concurrency (throttle knob; in-process bypasses HTTP rate limiter),
persisted in JSON.
- Confirmed `acceptFormat:'avif'` in a direct call equals browser `Accept: image/avif`.
## 2026-10-25 — diagnostics + safety + autonomy (v3)
- **Diagnostics / manual generation** (F06): `POST /generate` (one file, dry-run supported),
`GET /resolve-plan` (resolved transforms + predicted filename, no write), `GET /ops` (ring buffer).
- **Safe testing** (F02): staging compose stack + deploy/rollback scripts; fail-open, `enabled` default false.
- **Autonomous remote feedback**: staging-only, least privilege (scoped token, no prod Docker socket,
restricted SSH, private network, human-only prod promotion).
## 2026-10-25 — implementation + local verification (autonomous session)
Built the full extension locally and smoke-tested against an **isolated** Directus 11.17.4 stack
(`dev/compose.dev.yml`, port 8057, own pg + volumes). No contact with the prod instance.
### Implemented
- `src/lib/`: settings (env + JSON), state (`state.jsonl`, run state, lock, cancel flag),
presets (pinned system copy + custom merge, controller parity), warmer (in-process `getAsset` with
`acceptFormat` variants, concurrency, per-task error isolation, stream release), window, storage
(dynamic `getStorage` + local-FS fallback), diagnostics (resolve-plan prediction + ops ring buffer),
backfill (paginated, resumable, window-aware, idempotent).
- `src/endpoint/index.ts`: `/status`, `/config` (GET/PUT), `/presets`, `/run`, `/run/cancel`,
`/disk-usage`, `/generate`, `/resolve-plan`, `/ops`; admin-only guard.
- `src/hook/index.ts`: `files.upload` → fire-and-forget warm.
- `src/module/`: admin module (progress bar, disk usage, config form, diagnostics panel, ops table).
- `dev/`: compose stack, `sync-extension.sh`, seed images.
### Bugs found & fixed during smoke test
1. **Directus coerces env values** (`"true"` → boolean `true`, `"1"` → number). My string-only parsers
threw `e.toLowerCase is not a function`, which silently broke the upload hook. Fixed `settings.ts`
to accept `unknown` and stringify. Regression test added.
2. **State dir EACCES.** `/directus/extensions/.image-previews` (named volume on a non-existent path)
is root-owned; the API runs as `node`. This broke config writes and the backfill lock. Fixed: default
state dir is now `<STORAGE_LOCAL_ROOT>/.image-previews` (i.e. `/directus/uploads/.image-previews`),
which is writable; `IMAGE_PREVIEWS_STATE_DIR` still overrides.
3. **Bind-mount reload gotcha.** `rm -rf` of the bind-mounted extension dir breaks the Docker mount
(Directus removes the extension and never re-adds it). `sync-extension.sh` now updates in place;
restart the dev container after a sync.
### Verified evidence (local stack)
- `npm run typecheck` → 0 errors. `npm test` → **32/32 pass**. `npm run build` → `dist/app.js` + `dist/api.js`.
- Directus logs: `Loaded extensions: directus-extension-generate-image-previews` (no errors).
- `/status` → `{enabled:true, inWindow:true, locked:false}`; `/presets` → 6 system presets.
- Upload a JPEG → hook warmed **12 files** (6 presets × avif/webp) in `/directus/uploads`.
- **HASH-MATCH GATE PASSED:** after deleting the cached files, a real
`GET /assets/<id>?key=system-medium-cover` with `Accept: image/avif` produced exactly
`<id>__f15aa4b815722867859d2a3fa46ab50834dfc8dc.avif` — byte-identical to `resolve-plan`'s prediction.
- `/generate` dry-run reports `exists:true/false` correctly; `/ops` shows per-task timing.
- Backfill: `total=36` (3 images × 6 presets × 2 formats), `generated=36`, state `state.jsonl` = 36 lines.
Second run: `generated=0, hits=36`, state unchanged, file count unchanged → **idempotent/resumable**.
- Window gating: `/run` outside `03:00–05:00` → `409 outside-window` with `nextWindow`.
- `/disk-usage` → `157633 bytes / 36 files (source: storage)`.
- Cold vs warm (`system-medium-cover`): 114 ms → 16–30 ms. (`system-large-cover`): 304 ms → 36 ms.
### Decisions recorded
- System presets: version-pinned copy in `src/lib/system-presets.ts` (verbatim from `api/src/constants.ts`
v11.17.4). No `@directus/api` import needed for presets.
- Storage/disk usage: `@directus/api/storage/index` loaded via dynamic import (hidden from the bundler),
with a local-FS fallback; shares the host `getStorage()` singleton.
- New-preset trigger: **manual** via `POST /run` (idempotent) + the module's Run button; no
`settings.update` auto-trigger (avoids heavy background runs on unrelated settings changes).
- Scheduling: external cron → `POST /image-previews/run` (default).
- State/config dir: `<STORAGE_LOCAL_ROOT>/.image-previews` (writable), override with `IMAGE_PREVIEWS_STATE_DIR`.
### Additional verification (same session)
- `GET /extensions` → bundle registered with entries `module`, `hook`, `endpoint`, all enabled.
- Inert test: with `enabled:false`, uploading an image produced **zero** new preview files (25 → 25).
- Added `dev/compose.staging.yml`, `dev/deploy-staging.sh`, `dev/rollback-staging.sh` (templates;
NOT executed against any remote host).
- Added `storage.test.ts`; total tests now **34/34**.
- Restarted the dev stack after the final build; `/status`, `/disk-usage`, and bundle entries all good.
### Still open
- Admin module UI not yet verified in a real browser (bundle + entries registered; rendering/polling untested).
- Cancel + concurrent-run 409 not exercised end-to-end (unit-tested only; local runs finish too fast).
- Staging deploy not executed (scripts/templates provided); prod rollout not done.
- F01/F02/F07/F10 remain `passes:false` (F01 publish step, F02 staging run, F07 browser, F10 rollout).
## 2026-10-25 — feedback round (UI, defaults, custom formats, docs)
- **Defaults changed**: `enabled` now `true`; maintenance window defaults to `01:00–04:00`.
- **Custom fixed-format presets verified**: `custom-png` (format `png`) generated `.png` and
`resolve-plan` predicted the exact filename (`exists:true`). `custom-auto` generated avif+webp.
An unsupported preset (`format: bmp`) failed with Directus's clear error
("Expected one of: heic, heif, avif, jpeg, … received bmp") and is surfaced in Recent operations / `/ops`.
- **Effective format fix**: ops/presets now report the preset's real output format (`png`) instead of
`null`; backfill resume keys use the effective format (fixed a re-warm mismatch).
- **Progress accounting fix**: already-cached tasks now count toward `done`/`hits`. Runs report
`128/128` (120 hits + 8 expected failures) instead of `8/128`. After removing the unsupported preset:
`120/120 hits, 0 failed, 0 generated`.
- **Storage reporting**: `/disk-usage` + panel now split **previews** vs **originals** vs **total**
(raw + human): e.g. previews 774.7 KB / 115 files, originals 24.9 MB / 9 files, total 25.6 MB / 124.
- **Human-readable fields** added to `/status` (ISO + human dates, `etaHuman`, `rateHuman`),
`/disk-usage` (`bytesHuman`), `/ops` (`tsHuman`, `msHuman`), `/generate` (`msHuman`).
- **Admin module rewritten with Directus UI kit** (`v-button`, `v-input`, `v-checkbox`, `v-select`,
`v-notice`, `v-progress-linear`), field explanations, preset table with effective formats + shadow
warnings, storage table, error column in Recent operations, and a full collapsible **API reference**.
- **Run now** now ignores the maintenance window; **Cancel** only renders while a run is active.
- **Dimension vs file size clarified** in README/goal/AGENTS: the 6000 limit is a pixel-dimension guard in
Directus, not a file-size limit; the extension imposes none.
- Tests: **40/40** pass; typecheck clean; build OK.
### Still open
- Admin module UI not yet verified in a real browser (bundle + entries registered; rendering/polling untested).
- Cancel + concurrent-run 409 not exercised end-to-end.
- Staging deploy not executed (scripts/templates provided); prod rollout not done.
- F01/F02/F07/F10 remain `passes:false`.

52
AGENTS.md
View file

@ -5,11 +5,14 @@ Common confusion points for agents working here. Read before touching the warmer
## What this is ## What this is
A Directus **bundle** extension that pre-generates image transform ("preview") cache files so the A Directus **bundle** extension that pre-generates image transform ("preview") cache files so the
admin UI never pays the first-request sharp cost. Target server: **Directus 11.17.4** on `land-nest` admin UI never pays the first-request sharp cost.
(self-hosted, PM2 cluster, `STORAGE_LOCAL_ROOT=./uploads`).
Bundle entries: an api **hook** (`files.upload`), an api **endpoint** (`/image-previews/*`), and an Bundle entries: an api **hook** (`files.upload`), an api **endpoint** (`/image-previews/*`), and an app
app **module** (`image-previews` admin page: progress, disk usage, config). **module** (`image-previews` admin page: progress, storage, config, diagnostics, API reference).
Tested against Directus **11.17.4** (self-hosted, PM2 cluster, local disk storage). The manifest targets
`^11.0.0`; the extension uses non-public `@directus/api` internals at runtime, so re-verify on each
Directus minor upgrade.
## Directus transform invariants (source-verified, tag v11.17.4) — DO NOT REGRESS ## Directus transform invariants (source-verified, tag v11.17.4) — DO NOT REGRESS
@ -71,7 +74,8 @@ const { stream } = await assets.getAsset(fileId, {
``` ```
`acceptFormat` reproduces what a browser's `Accept` header does → byte-identical cache filename with `acceptFormat` reproduces what a browser's `Accept` header does → byte-identical cache filename with
no HTTP/token. Warm **both** `avif` and `webp` for `format:'auto'` presets. no HTTP/token. Warm **both** `avif` and `webp` for `format:'auto'` presets. For fixed-format presets,
omit `acceptFormat` and let the preset's own format drive.
HTTP-replay (`GET /assets/{id}?key=…`) is the documented **fallback** only (needs a token + rate-limit HTTP-replay (`GET /assets/{id}?key=…`) is the documented **fallback** only (needs a token + rate-limit
pacing). pacing).
@ -91,39 +95,19 @@ pacing).
silently killed the upload hook. silently killed the upload hook.
- **Docker bind mounts**: never `rm -rf` the bind-mounted extension directory — it breaks the mount and - **Docker bind mounts**: never `rm -rf` the bind-mounted extension directory — it breaks the mount and
Directus removes the extension without re-adding it. Sync contents in place; restart the dev container. Directus removes the extension without re-adding it. Sync contents in place; restart the dev container.
- The dev stack (`dev/compose.dev.yml`) is the only place to test; never load this into prod while - **Dimension vs file size**: `ASSETS_TRANSFORM_IMAGE_MAX_DIMENSION` (default 6000) is a **pixel
iterating (prod has `EXTENSIONS_AUTO_RELOAD=true`). dimension** guard in Directus, not a file-size limit. The extension imposes no limit; raise the
Directus env var to process larger images.
- **Unsupported preset formats** (e.g. `bmp`) fail at generation; surface the error in `/ops` and the
## Server facts (land-nest) module's Recent operations.
- The dev stack (`dev/compose.dev.yml`) is the only place to test; never load this into a production
- Directus image `directus/directus:11.17.4`; `PUBLIC_URL=https://data.landshaft.life`; instance while iterating (production may have `EXTENSIONS_AUTO_RELOAD=true`).
`PM2_INSTANCES=8`; `STORAGE_LOCAL_ROOT=./uploads`; logs at `~/apps/directus/logs`.
- `ASSETS_TRANSFORM_MAX_CONCURRENT=45` per worker → over it errors; catch per file/preset.
- `ASSETS_TRANSFORM_IMAGE_MAX_DIMENSION` default 6000 is a **pixel dimension** guard, not a file-size
limit. The extension imposes no limit; raise the Directus env var to process larger images.
- Local storage writes are **non-atomic** — warm only after an upload is committed.
- `RATE_LIMITER_*` applies to HTTP only → irrelevant to in-process generation (this is why the admin
knob is *concurrency*, not a rate limit).
- Observed bottleneck (2026-10-23 log triage): `storage:local:responseTime 1206ms`,
`pg:responseTime` 150–738ms; **no** 503s, **no** sharp errors. Pre-generation removes the
cold-transform cost; uploads-volume I/O latency is a separate ops concern.
## Safety (do not crash prod)
- Every entrypoint body is wrapped so thrown errors are logged and swallowed — never propagate into
Directus core. ALL behaviour is gated by `enabled` (default **false**): a loaded-but-disabled bundle
does nothing.
- Test only on the **staging** stack (`dev/compose.dev.yml`), never the prod instance.
- Rollout = backup previous `dist/` → deploy → healthcheck → enable; rollback = restore `dist/` + disable.
- Automation (autonomous loops) gets **staging only**: a scoped token limited to `/image-previews/*`
(+ `directus_files` metadata read), never a prod admin token; never the prod Docker socket; restricted
non-sudo SSH or opencode-on-host. Promotion to prod is a human action.
## Conventions ## Conventions
- Build via `@directus/extensions-sdk` (`npm run build` / `dev` / `link`); bundle emits `dist/app.js` + `dist/api.js`. - Build via `@directus/extensions-sdk` (`npm run build` / `dev` / `link`); bundle emits `dist/app.js` + `dist/api.js`.
- Deploy with `npm run publish` (scp `dist` + `package.json` to land-nest extensions dir). - Deploy by copying `dist/` + `package.json` into the Directus extensions dir (see README); a staging
helper lives in `dev/deploy-staging.sh`.
- Tests: `node:test` via `tsx` (`npm test`). Typecheck: `npm run typecheck`. - Tests: `node:test` via `tsx` (`npm test`). Typecheck: `npm run typecheck`.
- Do not add comments unless asked. Follow the sibling extensions' style. - Do not add comments unless asked. Follow the sibling extensions' style.
- Never commit without explicit request. - Never commit without explicit request.

82
README.md
View file

@ -8,29 +8,55 @@ A Directus **bundle**:
- **upload hook** — warm previews right after an image upload. - **upload hook** — warm previews right after an image upload.
- **new-preset trigger** — `Run now` warms every preset (system + custom) for existing images. - **new-preset trigger** — `Run now` warms every preset (system + custom) for existing images.
- **backfill job** — resumable, window-aware sweep over the existing library. - **backfill job** — resumable, window-aware sweep over the existing library.
- **admin module** — progress, storage breakdown, configuration, diagnostics, and API reference. - **admin module** — progress, storage breakdown, configuration, diagnostics, and an API reference.
Target: self-hosted Directus **11.17.4** (PM2 cluster, local disk storage). ## Requirements
## Why this approach (read AGENTS.md too) - Directus **11.x** (tested on **11.17.4**).
- Node 18+.
- A **non-sandboxed** local extension (it uses Directus's internal services at runtime — see
Compatibility).
- Local disk storage is supported out of the box; other storage drivers work for disk reporting through
the Directus storage manager.
## Why this approach
The cache file is `<filename_disk>__<object-hash(resolved transforms)>.<ext>`. The hash depends on the The cache file is `<filename_disk>__<object-hash(resolved transforms)>.<ext>`. The hash depends on the
RESOLVED transforms, and every `system-*` preset is `format:'auto'` — which resolves differently RESOLVED transforms, and every `system-*` preset is `format:'auto'` — which resolves differently
server-side (jpg/png) than in a browser (avif). We therefore generate **in-process** with an explicit server-side (jpg/png) than in a browser (avif). This extension therefore generates **in-process** with an
`acceptFormat` (`avif` + `webp`), reproducing the browser's cache file byte-for-byte without any HTTP explicit `acceptFormat` (`avif` + `webp`), reproducing the browser's cache file byte-for-byte without any
request. Naive warming (or the popular community extension) silently misses. HTTP request. Naive warming (and several community extensions) silently misses because of this.
## Install / build / publish ## Install
```bash ```bash
git clone https://code.at.locus.im/Rawman/directus-extension-generate-image-previews.git
cd directus-extension-generate-image-previews
npm install npm install
npm run build
```
Copy the build into your Directus extensions directory:
```bash
cp -r dist package.json /path/to/directus/extensions/directus-extension-generate-image-previews/
```
Restart Directus (or let `EXTENSIONS_AUTO_RELOAD` pick it up). The **Image Previews** module appears in
the admin sidebar.
## Development
```bash
npm run build # emits dist/app.js + dist/api.js npm run build # emits dist/app.js + dist/api.js
npm run typecheck # 0 errors npm run typecheck # 0 errors
npm test # unit tests npm test # unit tests
npm run dev # watch build npm run dev # watch build
npm run publish # scp dist + package.json to the (staging) deploy target npm run link # link into a local Directus extensions dir
``` ```
An isolated Directus 11.17.4 stack for smoke testing lives in `dev/` — see `dev/README.md`.
## Settings ## Settings
Defaults are applied automatically; everything is editable at runtime from the admin module — no rebuild. Defaults are applied automatically; everything is editable at runtime from the admin module — no rebuild.
@ -55,6 +81,8 @@ Defaults are applied automatically; everything is editable at runtime from the a
`format:'auto'` follows `formats`. `format:'auto'` follows `formats`.
- A custom preset whose `key` equals a system key is **shadowed** (system wins, same as Directus) and is - A custom preset whose `key` equals a system key is **shadowed** (system wins, same as Directus) and is
flagged in the admin module. flagged in the admin module.
- Unsupported format strings (e.g. `bmp`) fail at generation; the error is shown in Recent operations
and `GET /image-previews/ops`.
## Storage reporting ## Storage reporting
@ -63,7 +91,7 @@ Defaults are applied automatically; everything is editable at runtime from the a
## API endpoints ## API endpoints
All require an admin session or admin token. Human-readable fields accompany machine-readable ones. All require a Directus admin session or admin token. Human-readable fields accompany machine-readable ones.
- `GET /image-previews/status` — run progress, rate, ETA, window state, ISO + human dates - `GET /image-previews/status` — run progress, rate, ETA, window state, ISO + human dates
- `GET /image-previews/config`, `PUT /image-previews/config` - `GET /image-previews/config`, `PUT /image-previews/config`
@ -77,16 +105,27 @@ All require an admin session or admin token. Human-readable fields accompany mac
The admin module includes a full API reference with curl examples. The admin module includes a full API reference with curl examples.
## Scheduling
Scheduled runs are window-bound; an external cron can trigger them:
```bash
0 * * * * curl -fsS -X POST http://localhost:8055/image-previews/run \
-H 'Authorization: Bearer <admin-token>' -H 'Content-Type: application/json' -d '{}'
```
`Run now` in the admin module ignores the window.
## Safe testing & deployment ## Safe testing & deployment
Never test against the prod instance. Never test against a production instance.
- **Local / staging** (`dev/compose.dev.yml`): Directus 11.17.4 + its own postgres + volumes, on separate - **Local / staging** (`dev/compose.dev.yml`): Directus 11.17.4 + its own postgres + volumes, on separate
ports. Shares nothing with prod. See `dev/README.md`. ports. Shares nothing with production. See `dev/README.md`.
- **Fail-open**: entrypoints never throw into Directus core. - **Fail-open**: entrypoints never throw into Directus core.
- **Rollout**: build → verify on staging → back up previous `dist/` → deploy → healthcheck → enable. - **Rollout**: build → verify on staging → back up the previous `dist/` → deploy → healthcheck → enable.
**Rollback**: restore previous `dist/`. **Rollback**: restore the previous `dist/`.
- Scripts: `dev/deploy-staging.sh`, `dev/rollback-staging.sh`, template `dev/compose.staging.yml`. - Helpers: `dev/deploy-staging.sh`, `dev/rollback-staging.sh`, template `dev/compose.staging.yml`.
## Caveats ## Caveats
@ -94,9 +133,16 @@ Never test against the prod instance.
width or height exceeds `ASSETS_TRANSFORM_IMAGE_MAX_DIMENSION` (default **6000 px**). This is a Directus width or height exceeds `ASSETS_TRANSFORM_IMAGE_MAX_DIMENSION` (default **6000 px**). This is a Directus
env var, not something this extension imposes. To process larger images, raise it on the Directus env var, not something this extension imposes. To process larger images, raise it on the Directus
instance (and watch memory). Upload size limits (`FILES_MAX_UPLOAD_SIZE`, `MAX_PAYLOAD_SIZE`) are separate. instance (and watch memory). Upload size limits (`FILES_MAX_UPLOAD_SIZE`, `MAX_PAYLOAD_SIZE`) are separate.
- Warm only after uploads are committed (local writes are non-atomic). - Warm only after uploads are committed (local storage writes are non-atomic).
- In-process generation bypasses the HTTP rate limiter; bound it with `concurrency` + the window. - In-process generation bypasses the HTTP rate limiter; bound it with `concurrency` + the window.
- Unsupported custom preset formats (e.g. `bmp`) fail at generation time; the error appears in - PM2 cluster: shared state is file-backed, so all workers see the same run state.
Recent operations / `/ops`.
See `.opencode/goal.md` for the full contract and `.opencode/feature_list.json` for the task list. ## Compatibility
Tested against Directus **11.17.4**. The extension reads Directus's own transform pipeline and storage
manager (`@directus/api/utils/transformations`, `@directus/api/storage/index`) via runtime imports; these
are **not a public API** and may change between Directus versions. Re-verify after a Directus upgrade,
especially the hash-match gate (`resolve-plan` vs a real browser asset request).
The extension is not sandboxed: Directus's sandbox forbids importing anything except `directus:api`, which
this extension needs for storage/transform access.

22
SECURITY.md Normal file
View file

@ -0,0 +1,22 @@
# Security
## Access model
- Every `/image-previews/*` endpoint requires an authenticated **Directus admin** (session or admin
token). Non-admin requests receive `403`.
- The admin module is only reachable from the Directus admin app.
- There is no separate token mechanism and no unauthenticated write path.
## Operational guidance
- Do not expose a production admin token to automation. For autonomous/remote workflows, use a staging
instance and least-privilege access.
- The extension generates transforms in-process; it does not accept image URLs or arbitrary file paths
from clients.
- Persisted state lives under `<STORAGE_LOCAL_ROOT>/.image-previews` and contains no secrets.
- Uploaded file contents are never logged; only file ids, preset keys, formats, timings, and error
messages.
## Reporting a vulnerability
Please report security issues privately to **rawman@locus.im** rather than opening a public issue.

2
dev/compose.staging.yml
View file

@ -6,7 +6,7 @@
# #
# Keep ports/volumes/network distinct from the prod stack. # Keep ports/volumes/network distinct from the prod stack.
name: land-nest-staging name: directus-staging
services: services:
db: db:

136
package.json
View file

@ -1,63 +1,77 @@
{ {
"name": "directus-extension-generate-image-previews", "name": "directus-extension-generate-image-previews",
"description": "Pre-generate Directus image transforms: on upload, on demand, and via a scheduled resumable backfill, with an admin UI (progress, disk usage, maintenance window)", "description": "Pre-generate Directus image transforms: on upload, on demand, and via a scheduled resumable backfill, with an admin UI (progress, storage, maintenance window, diagnostics)",
"author": { "author": {
"name": "Rawman Grey", "name": "Rawman Grey",
"email": "rawman@locus.im", "email": "rawman@locus.im",
"url": "https://locus.im" "url": "https://locus.im"
}, },
"icon": "extension", "repository": {
"version": "0.1.0", "type": "git",
"keywords": [ "url": "https://code.at.locus.im/Rawman/directus-extension-generate-image-previews.git"
"directus", },
"directus-extension", "homepage": "https://code.at.locus.im/Rawman/directus-extension-generate-image-previews",
"directus-extension-bundle" "bugs": {
], "url": "https://code.at.locus.im/Rawman/directus-extension-generate-image-previews/issues"
"type": "module", },
"files": [ "icon": "extension",
"dist" "version": "0.1.0",
], "keywords": [
"directus:extension": { "directus",
"type": "bundle", "directus-extension",
"path": { "directus-extension-bundle",
"app": "dist/app.js", "images",
"api": "dist/api.js" "thumbnails",
}, "previews",
"entries": [ "transformations"
{ ],
"type": "module", "type": "module",
"name": "image-previews", "files": [
"source": "src/module/index.ts" "dist"
}, ],
{ "directus:extension": {
"type": "hook", "type": "bundle",
"name": "image-previews-hook", "path": {
"source": "src/hook/index.ts" "app": "dist/app.js",
}, "api": "dist/api.js"
{ },
"type": "endpoint", "entries": [
"name": "image-previews-endpoint", {
"source": "src/endpoint/index.ts" "type": "module",
} "name": "image-previews",
], "source": "src/module/index.ts"
"host": "^10.10.0 || ^11.0.0" },
}, {
"scripts": { "type": "hook",
"build": "directus-extension build", "name": "image-previews-hook",
"dev": "directus-extension build -w --no-minify", "source": "src/hook/index.ts"
"link": "directus-extension link", },
"typecheck": "tsc --noEmit", {
"test": "node --import tsx --test src/lib/*.test.ts", "type": "endpoint",
"publish": "scp -r ./dist ./package.json land-nest:/home/oak/apps/directus/extensions/directus-extension-generate-image-previews/" "name": "image-previews-endpoint",
}, "source": "src/endpoint/index.ts"
"devDependencies": { }
"@directus/extensions-sdk": "13.0.1", ],
"@types/node": "^22.13.5", "host": "^11.0.0"
"tsx": "^4.19.2", },
"typescript": "^5.7.3", "engines": {
"vue": "^3.5.13" "node": ">=18"
}, },
"dependencies": { "scripts": {
"object-hash": "^3.0.0" "build": "directus-extension build",
} "dev": "directus-extension build -w --no-minify",
"link": "directus-extension link",
"typecheck": "tsc --noEmit",
"test": "node --import tsx --test src/lib/*.test.ts"
},
"devDependencies": {
"@directus/extensions-sdk": "13.0.1",
"@types/node": "^22.13.5",
"tsx": "^4.19.2",
"typescript": "^5.7.3",
"vue": "^3.5.13"
},
"dependencies": {
"object-hash": "^3.0.0"
}
} }