120 lines
6.7 KiB
Markdown
120 lines
6.7 KiB
Markdown
---
|
||
name: directus-image-previews
|
||
description: Work on the directus-extension-generate-image-previews bundle (Directus 11 image transform pre-generation). Use when changing the warmer, backfill, scheduler, admin module, endpoints, presets, deployment, or when debugging preview generation in this repo.
|
||
---
|
||
|
||
# directus-image-previews — working rules
|
||
|
||
Read this before changing the extension. It encodes hard-won invariants; breaking them silently
|
||
produces cache misses or production incidents. For general Directus-extension patterns, see the global
|
||
`directus-extensions` skill.
|
||
|
||
## Scope
|
||
|
||
A Directus **bundle** (tested on **11.17.4**, manifest `host: ^11.0.0`) that pre-generates image transform
|
||
cache files:
|
||
|
||
- `src/hook/` — `files.upload` warmer + in-process nightly `schedule('*/15 * * * *')`.
|
||
- `src/endpoint/` — `/image-previews/*` admin API.
|
||
- `src/module/` — admin page (progress, storage, config, diagnostics, API reference).
|
||
- `src/lib/` — settings, state, presets, warmer, backfill, diagnostics, storage, window, format.
|
||
- `dev/` — isolated Directus 11.17.4 stack + deploy/rollback scripts.
|
||
|
||
## Invariants — do not regress
|
||
|
||
1. **Cache filename** = `<basename(filename_disk)>__<object-hash(resolved transforms)>.<ext>`, beside the
|
||
original in the storage root.
|
||
2. **Hash = `object-hash@3` defaults** (sha1/hex, `unorderedArrays:false` → array order matters,
|
||
`unorderedObjects:true`), over the **resolved** transform array.
|
||
3. **`resolvePreset` order**: `transforms[]` → `toFormat` (if `format`/`quality`) → `resize` (+`extract`).
|
||
4. **Never pass `{ key }` alone** — `resolvePreset` ignores `key`. Always pass the full preset object.
|
||
5. **`format:'auto'` needs an explicit `acceptFormat`** (`avif`/`webp`); otherwise server-side resolves to
|
||
jpg/png and the hash won't match the browser.
|
||
6. **Preset registry mirrors the controller**: `[...SYSTEM_PRESETS, ...storage_asset_presets]`, first match
|
||
wins. `GET /settings` returns only custom presets.
|
||
7. **`AssetsService.getAsset` awaits the write**; destroy/unpipe the returned stream to free the FD.
|
||
8. **Local storage writes are not atomic** — never generate the same file from two workers concurrently.
|
||
9. **State dir** is `<STORAGE_LOCAL_ROOT>/.image-previews` (writable). Do not use
|
||
`/directus/extensions/...` (root-owned → EACCES).
|
||
10. **Env values from the Directus context are coerced** (booleans/numbers). Stringify before parsing.
|
||
11. **All authoritative state is file-backed** (PM2 cluster). Never in-memory.
|
||
12. **Hooks never throw into Directus core**; all behavior is gated by `enabled`.
|
||
|
||
## Presets / formats semantics
|
||
|
||
- System presets are all `format:'auto'` → one file per configured `formats` (default `avif,webp`).
|
||
- Custom preset with a fixed `format` → exactly that format, `formats` is ignored.
|
||
- Custom preset with `auto` → follows `formats`.
|
||
- Custom preset with no `format`/`quality` → resize-only (original extension, still hashed).
|
||
- A custom key equal to a system key is **shadowed** (system wins) and flagged by `/presets`.
|
||
|
||
## Backfill / scheduler semantics
|
||
|
||
- `Run now` → `POST /run` with `ignoreWindow:true`; runs until cancelled.
|
||
- Nightly → `schedule('*/15 * * * *')`: only inside the maintenance window, no active run, and not already
|
||
completed today. It starts/resumes the backfill.
|
||
- The backfill **pauses** when the window closes (`run.paused=true`) and resumes next night.
|
||
- Pre-scans storage filenames and counts existing preview files as `already cached` (skips generation).
|
||
- `total = image files × tasksPerFile`; `done` counts cached + generated + failed.
|
||
- One run at a time (in-process guard + `run.lock`).
|
||
|
||
## State files (on the Directus host/container)
|
||
|
||
```
|
||
<STORAGE_LOCAL_ROOT>/.image-previews/
|
||
settings.json effective config
|
||
run.json progress (running/done/total/generated/hits/failed/rate/eta/paused/cancelled/error)
|
||
state.jsonl completed {file_id,preset_key,format} → resume
|
||
failures.jsonl append-only failure events; replay = latest per task; compacted under run.lock
|
||
run.lock cluster-wide run lock
|
||
cancel.flag cancellation request
|
||
```
|
||
|
||
## Debugging
|
||
|
||
- `GET /image-previews/status` — progress, window, paused, last error.
|
||
- `GET /image-previews/presets` — effective formats, shadowed keys.
|
||
- `GET /image-previews/resolve-plan?fileId=&key=` — predicted cache filename, no write. **Use this first.**
|
||
- `POST /image-previews/generate` `{fileId,keys?,formats?,dryRun?}` — one file, per-task results.
|
||
- `GET /image-previews/ops?limit=` — recent operations with errors (e.g. unsupported preset format).
|
||
- Directus log (file transport): `Added extensions`, `Extensions reloaded`, `image-previews: …`.
|
||
- Common failure: custom preset format not supported by sharp (e.g. `bmp`) → error in `/ops`; fix the preset.
|
||
|
||
## Verification gate
|
||
|
||
Before shipping a change that touches generation, prove:
|
||
|
||
1. `resolve-plan` filename for `?key=system-medium-cover` is byte-identical to the file produced by a real
|
||
browser request with `Accept: image/avif`.
|
||
2. `npm test` green, `npm run typecheck` clean, `npm run build` emits both bundles.
|
||
3. Isolated `dev/` stack: upload → 12 files (6 presets × 2 formats), backfill idempotent, pause/cancel works.
|
||
|
||
## Build / deploy
|
||
|
||
- **Never build on the production host** (no Node; npm is unreliable there).
|
||
- Build locally: `npm ci && npm run build && npm test`.
|
||
- Package `dist/` + `package.json` (+ README/LICENSE) into a tarball; SCP it.
|
||
- Install: copy `dist` **first**, `package.json` **last**; back up the previous `dist`.
|
||
- `EXTENSIONS_AUTO_RELOAD=true` in prod — never copy a half-built extension into the watched dir. If a
|
||
reload looks ambiguous, restart the container.
|
||
- Rollback: restore the backed-up `dist` or remove the extension directory.
|
||
- Public repo: `git@code-at-locus.im:Rawman/directus-extension-generate-image-previews.git`, branch `master`.
|
||
`.opencode/` internal notes are untracked; only `.opencode/skills/` is tracked.
|
||
|
||
## Checklist for any change
|
||
|
||
- [ ] Invariants above untouched (or explicitly justified).
|
||
- [ ] Tests added/updated; 43+ green; typecheck clean.
|
||
- [ ] `resolve-plan` hash-match still proven.
|
||
- [ ] No new in-memory state; state dir still writable.
|
||
- [ ] Endpoint stays admin-guarded; no secrets/file contents logged.
|
||
- [ ] Deploy payload is `dist` + `package.json`; backup taken; reload verified.
|
||
- [ ] README/AGENTS updated if behavior or settings changed.
|
||
|
||
## Links
|
||
|
||
- Global skill: `directus-extensions` (bundles, context APIs, storage, transforms, deployment).
|
||
- Docs: https://directus.com/docs/guides/extensions/overview
|
||
- Transforms: https://directus.com/docs/guides/files/transform
|
||
- Source anchors: `api/src/services/assets.ts`, `api/src/utils/transformations.ts`,
|
||
`api/src/controllers/assets.ts`, `api/src/constants.ts`.
|