AI Implementation feature(917): Category Icons (#65)
This commit was merged in pull request #65.
This commit is contained in:
@@ -0,0 +1,180 @@
|
|||||||
|
# Implementation Plan: Job 917 — Category Icons
|
||||||
|
|
||||||
|
## 0. Status
|
||||||
|
|
||||||
|
**NOT IMPLEMENTED.** The repository contains zero PNG asset files anywhere under
|
||||||
|
`/repo` (confirmed via `find /repo -name "*.png"`). The database rows created by
|
||||||
|
`UpdateSystemCategoryKeys1700000000300` and
|
||||||
|
`RepairCategorySchemaAndSystemCategories1700000000400` reference
|
||||||
|
`/uploads/icons/{CRY,HW,MSC,PWN,REV,WEB}.png`, but nothing on bootstrap writes
|
||||||
|
those PNGs to `UPLOAD_DIR/icons`. `setup.sh` only `mkdir -p`s the directory; it
|
||||||
|
does not populate it. As a result `GET /uploads/icons/CRY.png` returns 404 and
|
||||||
|
the challenges board shows broken images.
|
||||||
|
|
||||||
|
The job must: generate the canonical six system-category PNG icons at
|
||||||
|
`UPLOAD_DIR/icons/{KEY}.png` on every startup, idempotently, so a fresh clone
|
||||||
|
that runs `setup.sh` and `npm start` has working category icons immediately.
|
||||||
|
|
||||||
|
## 1. Architectural Reconnaissance
|
||||||
|
|
||||||
|
- **Codebase style & conventions:** TypeScript strict, NestJS 10 modules, ESM
|
||||||
|
CommonJS hybrid, async/await everywhere. Migrations are forward-only classes
|
||||||
|
implementing `MigrationInterface`; service-layer logic in `*.service.ts`. A
|
||||||
|
recent precedent (Job 919 filesystem recovery) shows the codebase prefers
|
||||||
|
small, idempotent startup hooks in `DatabaseInitService` rather than changes
|
||||||
|
to individual migrations.
|
||||||
|
- **Data Layer:** TypeORM + `better-sqlite3`. DB at `DATABASE_PATH`
|
||||||
|
(default `./data/db.sqlite`). Upload files live at `UPLOAD_DIR`
|
||||||
|
(default `./data/uploads`) and are served by `app.use('/uploads',
|
||||||
|
express.static(uploadDir))` in `backend/src/main.ts:79`. Migrations are
|
||||||
|
registered in `backend/src/database/database.module.ts:40`.
|
||||||
|
- **Test Framework & Structure:** Jest with a multi-project config at
|
||||||
|
`tests/jest.config.js`. All backend specs live in `tests/backend/**.spec.ts`,
|
||||||
|
run via `npm test` (or `npm run test:backend`). All frontend specs live in
|
||||||
|
`tests/frontend/**.spec.ts`. New tests must follow the same layout — no
|
||||||
|
source-side spec files.
|
||||||
|
- **Required Tools & Dependencies:** No new dependencies are needed. `sharp`
|
||||||
|
(already a `backend` runtime dep at `^0.35.3`) provides PNG generation, and
|
||||||
|
the root `devDependencies` already pin `sharp` and `jest`. The existing
|
||||||
|
`admin-icon-normalize.spec.ts` test already verifies that the same sharp
|
||||||
|
pipeline produces a valid 128×128 PNG.
|
||||||
|
|
||||||
|
## 2. Impacted Files
|
||||||
|
|
||||||
|
- **To Modify:**
|
||||||
|
- `backend/src/database/database-init.service.ts` — call the new icon-seed
|
||||||
|
step after `runMigrations` so freshly migrated databases (and warm databases
|
||||||
|
that pre-date this job) both end up with canonical icons on disk.
|
||||||
|
- `setup.sh` — leave as-is (the directory is already created); no shell-side
|
||||||
|
asset copy is required because the generation will happen at startup.
|
||||||
|
|
||||||
|
- **To Create:**
|
||||||
|
- `backend/src/database/system-category-icons.ts` — small pure helper that
|
||||||
|
exports `CANONICAL_SYSTEM_ICON_KEYS` (the six canonical keys in fixed order:
|
||||||
|
`['CRY','HW','MSC','PWN','REV','WEB']`) and an async
|
||||||
|
`seedSystemCategoryIcons(uploadDir)` function. Re-importing this from the
|
||||||
|
migration that owns the canonical list would create a circular dependency
|
||||||
|
through `database.module.ts`, so the helper mirrors the canonical set
|
||||||
|
directly and references the migration file via comment for traceability.
|
||||||
|
It generates a deterministic 128×128 PNG for each key using `sharp` (solid
|
||||||
|
color background derived from a stable hash of the key, plus the 2–6-letter
|
||||||
|
abbreviation rendered in white) and writes
|
||||||
|
`<uploadDir>/icons/<KEY>.png` only if the file does not already exist
|
||||||
|
(idempotent — does not overwrite an admin-uploaded icon).
|
||||||
|
- `tests/backend/system-category-icons.spec.ts` — verifies that
|
||||||
|
`seedSystemCategoryIcons` (a) writes six PNGs into a temp
|
||||||
|
`icons/` directory, (b) writes files whose on-disk size is > 0 and whose
|
||||||
|
`sharp().metadata()` reports `format === 'png'`, `width === 128`,
|
||||||
|
`height === 128`, (c) is idempotent — running it twice does not change
|
||||||
|
file contents when the first run's bytes are preserved (assert by checking
|
||||||
|
`stat.mtimeMs` is unchanged or by capturing the buffer once and comparing),
|
||||||
|
(d) does not throw when the target directory is missing (it creates it),
|
||||||
|
and (e) covers exactly the canonical six keys (CRY, HW, MSC, PWN, REV,
|
||||||
|
WEB). One focused test per behavior above; no mocks of `sharp` (use a
|
||||||
|
real `os.tmpdir()`-based dir for the assertions).
|
||||||
|
|
||||||
|
## 3. Proposed Changes
|
||||||
|
|
||||||
|
1. **Helper module (`backend/src/database/system-category-icons.ts`):**
|
||||||
|
- `import * as fs from 'fs'` and `import * as path from 'path'` and
|
||||||
|
`import sharp from 'sharp'`.
|
||||||
|
- Export `const CANONICAL_SYSTEM_ICON_KEYS = ['CRY','HW','MSC','PWN','REV','WEB'] as const;`
|
||||||
|
with `export type CanonicalSystemIconKey = (typeof CANONICAL_SYSTEM_ICON_KEYS)[number];`
|
||||||
|
— the same six keys declared in
|
||||||
|
`1700000000400-RepairCategorySchemaAndSystemCategories.ts:13` as
|
||||||
|
`CANONICAL_SYSTEM_CATEGORIES`. The duplication is intentional to avoid a
|
||||||
|
migration → runtime import cycle; a code comment must point at the
|
||||||
|
migration file as the canonical source of truth so future jobs keep them
|
||||||
|
in sync.
|
||||||
|
- Define `export const SYSTEM_ICON_SIZE = 128` matching the existing
|
||||||
|
`uploads.controller.ts:88` resize target so generated icons look
|
||||||
|
consistent with admin-uploaded ones.
|
||||||
|
- Implement
|
||||||
|
`export async function seedSystemCategoryIcons(uploadDir: string): Promise<{ written: string[]; skipped: string[] }>`:
|
||||||
|
- Resolve `iconsDir = path.join(uploadDir, 'icons')`, create it
|
||||||
|
recursively if missing.
|
||||||
|
- For each `key` in `CANONICAL_SYSTEM_ICON_KEYS`, target file
|
||||||
|
`path.join(iconsDir, `${key}.png`)`.
|
||||||
|
- If the file already exists and is non-empty, push to `skipped`.
|
||||||
|
- Otherwise generate a 128×128 PNG via
|
||||||
|
`sharp({ create: { width, height, channels: 4, background: <color for key> } }).png().toBuffer()`
|
||||||
|
(color picked by a deterministic hash of `key` so each icon is visually
|
||||||
|
distinct but reproducible across machines/containers). Wrap in
|
||||||
|
`try/catch` and on failure `throw new Error('Failed to seed system
|
||||||
|
category icon ' + key + ': ' + err.message)`. On success `fs.writeFileSync`
|
||||||
|
the buffer to disk and push the absolute path to `written`.
|
||||||
|
- Return `{ written, skipped }`.
|
||||||
|
- Add a top-of-file comment noting that this helper must stay in sync with
|
||||||
|
`CANONICAL_SYSTEM_CATEGORIES` in
|
||||||
|
`1700000000400-RepairCategorySchemaAndSystemCategories.ts` and with the
|
||||||
|
`iconPath` URLs hard-coded in
|
||||||
|
`1700000000300-UpdateSystemCategoryKeys.ts:8-13` and
|
||||||
|
`1700000000400-RepairCategorySchemaAndSystemCategories.ts:19-54`.
|
||||||
|
|
||||||
|
2. **Bootstrap hook (`backend/src/database/database-init.service.ts`):**
|
||||||
|
- Add `import { seedSystemCategoryIcons } from './system-category-icons';`
|
||||||
|
near the existing top-of-file imports.
|
||||||
|
- In `init()`, after the `runMigrations` block and after `verifySeed()`,
|
||||||
|
call a new `await this.ensureSystemCategoryIcons()`. The call must run
|
||||||
|
on every startup (not gated on `pendingList.length > 0`) so warm
|
||||||
|
databases that pre-date this job also get the icons on next boot.
|
||||||
|
- Implement
|
||||||
|
`private async ensureSystemCategoryIcons(): Promise<void>`:
|
||||||
|
- `const uploadDir = path.resolve(this.config.get<string>('UPLOAD_DIR', './data/uploads'));`
|
||||||
|
- If `uploadDir === ':memory:'` or contains `:memory:` skip with a debug
|
||||||
|
log (matches how the module already short-circuits in-memory test DBs
|
||||||
|
in some places and avoids touching whatever a test has wired up).
|
||||||
|
- Otherwise `const result = await seedSystemCategoryIcons(uploadDir);` and
|
||||||
|
log: `System category icons: written=${result.written.length}
|
||||||
|
skipped=${result.skipped.length}`. A failure must `this.logger.warn`
|
||||||
|
and **not** abort startup (icons are recoverable and a missing icon
|
||||||
|
does not block HTTP traffic — mirroring the
|
||||||
|
`verifySeed` "best-effort" pattern at line 60).
|
||||||
|
|
||||||
|
3. **No changes to `setup.sh`.** The directory is already created there and
|
||||||
|
the runtime hook handles the asset creation. (Adding an out-of-band
|
||||||
|
`sharp` invocation in bash would require either bundling `sharp` into the
|
||||||
|
shell script or shelling out to `node -e` — both are heavier than the
|
||||||
|
in-process hook.)
|
||||||
|
|
||||||
|
4. **No changes to existing migrations.** The DB rows already reference the
|
||||||
|
right URLs; we only need the files to exist on disk.
|
||||||
|
|
||||||
|
## 4. Test Strategy
|
||||||
|
|
||||||
|
- **Target Unit Test File:** `tests/backend/system-category-icons.spec.ts`
|
||||||
|
(single file, ~30–50 lines, no test infrastructure beyond `os.tmpdir()`
|
||||||
|
and `fs.mkdtempSync`).
|
||||||
|
- **Mocking Strategy:** No mocks. Use real `sharp` (already a project dev
|
||||||
|
dep, validated by `tests/backend/admin-icon-normalize.spec.ts:1`) and a
|
||||||
|
fresh per-test temp directory under `os.tmpdir()` (e.g.
|
||||||
|
`fs.mkdtempSync(path.join(os.tmpdir(), 'hipctf-icons-'))`) that is removed
|
||||||
|
in `afterEach`. Asserting on real files via `fs.statSync` and
|
||||||
|
`sharp(buf).metadata()` is faster and more reliable than mocking the fs
|
||||||
|
module.
|
||||||
|
- **Cases (keep minimal, success-path only):**
|
||||||
|
1. `seedSystemCategoryIcons` writes six PNGs into a fresh temp dir and
|
||||||
|
each one is a valid 128×128 PNG with `format === 'png'`.
|
||||||
|
2. Calling it twice with the same dir is idempotent — the second call
|
||||||
|
reports all six as `skipped` and does not touch `mtimeMs` of the
|
||||||
|
existing files (this is the contract that protects admin-uploaded
|
||||||
|
icons: if an admin uploads a `CRY.png`, a later restart must not
|
||||||
|
overwrite it).
|
||||||
|
3. The helper creates the `icons/` subdirectory when the parent dir is
|
||||||
|
empty (covers the fresh-clone case the job explicitly calls out).
|
||||||
|
- All tests must run under `npm test` (single command from repo root) and
|
||||||
|
complete in well under a second on a warm cache. No UI, no visual
|
||||||
|
verification, no complex infrastructure.
|
||||||
|
|
||||||
|
## 5. Cross-cutting Notes
|
||||||
|
|
||||||
|
- The existing `admin-icon-normalize.spec.ts` already proves the `sharp`
|
||||||
|
pipeline at the size we need (128×128 PNG) and is the closest precedent for
|
||||||
|
the new spec — the new test should follow its style (real sharp, real
|
||||||
|
buffers, no jest mocks).
|
||||||
|
- Do **not** modify the canonical migration files. They are forward-only and
|
||||||
|
already correct; the bug is purely an asset-on-disk problem.
|
||||||
|
- Do **not** add new npm dependencies. `sharp` is sufficient.
|
||||||
|
- The `/data` persistent volume rule applies only if any test wants to share
|
||||||
|
state across runs — this job does not need to; per-test temp dirs are fine
|
||||||
|
and isolated.
|
||||||
@@ -1,69 +0,0 @@
|
|||||||
# Implementation Plan: Job 919 — Crash-Safe Admin Database Backup/Restore and Danger Zone Recovery
|
|
||||||
|
|
||||||
## 1. Architectural Reconnaissance
|
|
||||||
- **Status:** The requested Job is **not fully implemented**. Restore currently has an in-process rollback handle only; a SIGKILL during or after the multi-path swap can leave the database and upload directory in different generations, and startup has no recovery sweep for rollback artifacts. Relevant existing implementation is in `backend/src/modules/admin/system/restore.service.ts` and `backend/src/modules/admin/system/filesystem-transaction.service.ts`.
|
|
||||||
- **Codebase style & conventions:** TypeScript monorepo with a NestJS 10 backend, Angular 17 standalone frontend, TypeORM over `better-sqlite3`, synchronous local filesystem primitives for staged archive files and rename operations, async Nest services/controllers, Zod schemas through `ZodValidationPipe`, and `ApiError`/global exception-envelope handling. `main.ts` explicitly initializes the database before listening. Controllers are thin and delegate destructive work to services.
|
|
||||||
- **Data Layer:** File-backed SQLite configured through `DATABASE_PATH`, TypeORM/`DataSource` for the live application connection, migrations with WAL enabled by `DatabaseInitService`, and direct `better-sqlite3` use in restore for an offline candidate database. Application uploads live below `UPLOAD_DIR`; backups encode the complete upload tree as base64 entries. No database schema migration is required for this fix unless the implementer elects to persist a transaction journal; the preferred design uses a durable filesystem transaction manifest instead of application tables so recovery works before ordinary application access.
|
|
||||||
- **Test Framework & Structure:** Jest 29 with `ts-jest`, two root projects (`backend` Node and `frontend` jsdom), and all tests in the dedicated `/repo/tests` tree. Backend restore coverage is concentrated in `tests/backend/admin-system-restore-commit.spec.ts`; transaction behavior currently has no focused dedicated test. Tests must remain CLI-runnable through the root `npm test`, use isolated per-test filesystem/database roots, and avoid UI/manual verification. Add only focused tests for crash/restart recovery, rollback-artifact cleanup, and coherent post-recovery state.
|
|
||||||
- **Required Tools & Dependencies:** Existing Node/npm toolchain, TypeScript, Jest, `better-sqlite3`, NestJS, TypeORM, and Node `fs`/`path`/`crypto` APIs are sufficient. No new runtime package is required. Do not add a database or external queue. `setup.sh` already creates the normal upload directories, installs dependencies, rebuilds `better-sqlite3`, and builds both workspaces; update it only if the chosen implementation introduces a required OS utility (not expected). If using subprocess-based crash simulation, use the existing Node runtime and Jest rather than adding a global CLI.
|
|
||||||
|
|
||||||
## 2. Impacted Files
|
|
||||||
- **To Modify:**
|
|
||||||
- `backend/src/modules/admin/system/filesystem-transaction.service.ts` — replace the ephemeral, in-memory-only swap state with a durable transaction manifest/state machine; make recovery deterministic across process death, preserve pair ordering, and clean rollback/staged artifacts only after the transaction reaches a durable terminal state.
|
|
||||||
- `backend/src/modules/admin/system/restore.service.ts` — create and finalize a restore transaction through the durable transaction service, avoid treating the database and uploads as independently committed, and ensure candidate/WAL/SHM/staging artifacts are included in recovery and cleanup decisions.
|
|
||||||
- `backend/src/database/database-init.service.ts` — invoke startup recovery before migrations/normal schema verification and fail closed if a transaction cannot be safely resolved; ensure recovery runs before HTTP traffic is accepted.
|
|
||||||
- `backend/src/main.ts` — only if needed to expose the recovery call in the explicit bootstrap sequence; preserve the documented `DatabaseInitService.init()`-before-`listen()` ordering and do not serve uploads before recovery completes.
|
|
||||||
- `backend/src/config/env.schema.ts` — only if a configurable transaction-manifest/stale-transaction retention setting is introduced; otherwise no change is needed.
|
|
||||||
- `backend/src/common/utils/upload.ts` — only if shared helpers are needed for locating the transaction journal/staging root; reuse `resolveSystemStagingDir` and keep system staging outside public uploads.
|
|
||||||
- `tests/backend/admin-system-restore-commit.spec.ts` — extend the existing minimal restore tests for a simulated interruption between pair swaps and verify the database/upload generation is coherent after recovery.
|
|
||||||
- `tests/backend/filesystem-transaction.spec.ts` — create a focused unit/integration test file in the dedicated backend test folder for durable manifest transitions, recovery of interrupted swaps, idempotent cleanup, and refusal to guess when the manifest is incomplete.
|
|
||||||
- `tests/backend/database-init-recovery.spec.ts` — create a focused startup-recovery test that runs the initialization recovery path against isolated file-backed data and checks orphan rollback directories are removed or safely restored before normal startup.
|
|
||||||
- `docs/api/admin-system.md` — update restore semantics/error behavior if the externally observable contract changes, especially recovery guarantees and startup handling.
|
|
||||||
- `docs/guides/admin-system-page.md` — update operational guidance only if restore failure/startup recovery messaging or operator cleanup expectations change.
|
|
||||||
- **To Create:**
|
|
||||||
- `backend/src/modules/admin/system/transaction-recovery.service.ts` (recommended) — a small startup/recovery coordinator if keeping recovery logic out of `FilesystemTransactionService`; it should discover manifests, classify states, resolve only provably safe outcomes, and report/fail closed on ambiguous state.
|
|
||||||
- `tests/backend/filesystem-transaction.spec.ts` — dedicated focused transaction tests.
|
|
||||||
- `tests/backend/database-init-recovery.spec.ts` — dedicated focused startup recovery tests.
|
|
||||||
- A durable manifest file format under the configured system staging/data directory at runtime, not a committed fixture; no persistent repository data should be added outside code/tests. Test-generated databases and upload trees should be created in unique temporary directories, or under `/data` only when a successor job must reuse them.
|
|
||||||
|
|
||||||
## 3. Proposed Changes
|
|
||||||
Explain the exact implementation logic step-by-step:
|
|
||||||
|
|
||||||
1. **Database / Schema Migration:**
|
|
||||||
- Do not add application tables or migrations for the transaction protocol. The failure occurs during filesystem replacement and can happen before a database connection is usable; recovery metadata must therefore be independent of SQLite and durable on the same filesystem as the database/upload paths.
|
|
||||||
- Define a versioned manifest containing a transaction id, operation name, ordered swap pairs, absolute or validated repository-relative live/staged/rollback paths, intended generation, and an explicit phase such as `prepared`, `live-snapshots-created`, `replacements-installed`, `verified`, `committed`, or `rollback-required`.
|
|
||||||
- Write the manifest atomically (`temporary manifest` → flush/close as supported → rename into the canonical manifest path) before the first live path is moved. Update it atomically after every irreversible filesystem step. Use a unique transaction directory under `resolveSystemStagingDir`, never under the public upload tree, and include database sidecars (`-wal`, `-shm`) in the database replacement protocol or explicitly normalize/remove them so an old WAL cannot be paired with a new main DB.
|
|
||||||
- Persist enough information to distinguish these cases after a kill: all old paths available and replacements incomplete means restore the entire old generation; every replacement installed and candidate verification recorded means finish/retain the new generation; any mixed/ambiguous state means do not delete either generation and fail startup closed with an actionable log/error. The implementation must never simply delete an unknown rollback directory.
|
|
||||||
- Ensure durability ordering: manifest exists before mutation; each rename/copy result is recorded before advancing; the terminal commit marker is written only after both database and upload generations are present and verified; only then remove rollback and staged artifacts. Cleanup is best effort after the terminal marker and must not erase the only known recoverable generation.
|
|
||||||
|
|
||||||
2. **Backend Logic & APIs:**
|
|
||||||
- Refactor `FilesystemTransactionService.stageSwap()` into a crash-recoverable protocol while retaining its public rollback/commit behavior for existing callers. Keep pair processing deterministic and perform rollback in reverse order for in-process failures, but make the manifest authoritative so a process restart can continue recovery.
|
|
||||||
- Add a public startup recovery method (either on `FilesystemTransactionService` or the recommended `TransactionRecoveryService`) that scans only the configured transaction root for unfinished manifests and stale restore rollback artifacts. It must be idempotent: repeated startup attempts converge to the same coherent state and do not recreate or delete unrelated files.
|
|
||||||
- For a manifest whose old database and old upload tree are both available, restore both old paths as one generation. For a manifest whose new database and new upload tree are both fully installed and whose post-swap verification marker is durable, finalize the new generation and remove old rollback artifacts. For the observed hybrid case (old DB plus new uploads, or vice versa), select the old generation only when both old paths are present; otherwise select the fully verified new generation only when both new paths are present; otherwise fail closed rather than producing a third hybrid.
|
|
||||||
- Add explicit validation of the recovered database and upload generation before finalization. Database validation should open the selected SQLite file read-only, run the existing foreign-key/integrity checks and required-admin invariant, and account for WAL/SHM sidecars. Upload validation should ensure the selected tree is a complete directory replacement, not a merge; do not copy individual files from one generation into the other. Any orphaned rollback/staged directory associated with a transaction should be removed only after the selected generation is installed and the durable commit/rollback marker is written.
|
|
||||||
- Change `RestoreService.commitRestore()` to use the durable transaction API and remove unused `liveBackupDb` behavior. Candidate DB construction should remain offline and candidate verification should happen before the swap whenever possible; `afterSwap` must verify the selected live DB without relying on an already-open TypeORM connection that may still point at the old inode. The result should report success only after the transaction is durably committed.
|
|
||||||
- Handle process-level concurrency by preventing two destructive filesystem transactions from running simultaneously. Reuse the existing operation-in-progress/error conventions if present; otherwise add a small lock/manifest guard in the transaction root. A stale lock must be recoverable only through manifest inspection, not by blindly deleting it.
|
|
||||||
- Keep controller routes and confirmation-token contracts unchanged. Restore success should continue to revoke refresh sessions; restore rollback errors should continue to map to `SYSTEM_RESTORE_ROLLED_BACK`. If startup recovery cannot establish a coherent state, fail startup before `app.listen()` and log the transaction id/phase without exposing secrets or raw archive contents.
|
|
||||||
- Update backup traversal if necessary to exclude all transaction metadata/rollback directories, not just `.staging`, so an interrupted restore can never be re-archived as application upload content. The exclusion must be based on exact reserved names/prefixes and remain scoped to `UPLOAD_DIR`.
|
|
||||||
- No frontend feature work is required: the existing admin restore UI already reports `SYSTEM_RESTORE_ROLLED_BACK` and logs the user out after a successful restore. Only update frontend pure helpers/tests if the backend introduces a new user-visible error code; prefer retaining existing codes.
|
|
||||||
|
|
||||||
3. **Frontend UI Integration:**
|
|
||||||
- No component or route changes are planned. The existing `SystemService` calls validation and commit endpoints, and the admin system page already handles success/failure. Verify that any recovery failure remains represented by the existing restore-rollback/startup failure behavior; do not add visual tests or UI infrastructure.
|
|
||||||
|
|
||||||
4. **Operational/startup integration:**
|
|
||||||
- Invoke recovery before `DatabaseInitService.ensureWalMode()`, migrations, seed verification, static upload mounting, and `app.listen()`. The ordering is critical because opening the old/new database or serving uploads before resolving the manifest can recreate the mixed state.
|
|
||||||
- Preserve configured path safety: resolve `DATABASE_PATH`, `UPLOAD_DIR`, and staging paths once through `ConfigService`; reject manifests whose paths escape the configured data roots; never follow symlinks or accept arbitrary manifest paths from the request payload.
|
|
||||||
- Document that recovery is automatic on restart, that an ambiguous transaction prevents startup rather than silently deleting data, and that an operator must preserve the transaction directory for forensic/manual recovery.
|
|
||||||
|
|
||||||
## 4. Test Strategy
|
|
||||||
- **Target Unit Test File:** `tests/backend/filesystem-transaction.spec.ts` for the transaction state machine and recovery behavior; `tests/backend/database-init-recovery.spec.ts` for startup integration; extend `tests/backend/admin-system-restore-commit.spec.ts` with one end-to-end restore coherence assertion. Keep all tests in `/repo/tests/backend`, never alongside source.
|
|
||||||
- **Mocking Strategy:** Use unique file-backed temporary directories per test (or `/data` only for intentionally persistent successor-job fixtures), real Node filesystem operations, and small real SQLite databases created with `better-sqlite3`. Mock only unrelated Nest/DataSource boundaries and time/randomness where deterministic manifest names are needed. Do not mock the filesystem for the core recovery tests because the bug is specifically a crash/interrupted-rename condition. Avoid requiring a browser, UI, external service, Docker, or a large seeded database.
|
|
||||||
- **Focused cases:**
|
|
||||||
1. Successful restore writes a terminal commit marker and leaves exactly the restored DB/upload generation with no rollback or stage artifacts.
|
|
||||||
2. Simulated interruption after the database pair is swapped but before uploads are swapped leaves a durable manifest; startup recovery restores the old database and old uploads together, removes both new/staged artifacts, and never leaves the backup upload file in the live tree.
|
|
||||||
3. Simulated interruption after both pairs are swapped but before cleanup allows startup recovery to retain the complete new generation and remove old rollback artifacts.
|
|
||||||
4. The observed hybrid layout (old live DB plus new live uploads plus old rollback upload tree) is resolved to the old generation when both old rollback paths are available; assertions cover DB rows, referenced upload files, absence of backup-only orphan files, and cleanup of rollback directories.
|
|
||||||
5. A manifest with neither a complete old generation nor a verified complete new generation causes startup recovery to fail closed and preserves artifacts for operator recovery.
|
|
||||||
6. Recovery is idempotent when invoked twice and does not alter unrelated uploads or databases.
|
|
||||||
7. Existing no-admin restore rejection and successful restore coverage remain green, proving the new transaction protocol does not weaken the `LAST_ADMIN` invariant or existing API contract.
|
|
||||||
- **CLI execution:** The existing root `npm test` runs both Jest projects in one command. The implementer must run the focused backend tests and then root lint/typecheck commands if defined by the repository; no new dependency or test command should be required. If the test runner executes suites in parallel, each suite must use a unique filesystem root and close all SQLite handles/resources in `afterEach`/`afterAll`.
|
|
||||||
@@ -4,6 +4,7 @@ import { DataSource } from 'typeorm';
|
|||||||
import * as path from 'path';
|
import * as path from 'path';
|
||||||
import { FilesystemTransactionService } from '../modules/admin/system/filesystem-transaction.service';
|
import { FilesystemTransactionService } from '../modules/admin/system/filesystem-transaction.service';
|
||||||
import { resolveSystemStagingDir } from '../common/utils/upload';
|
import { resolveSystemStagingDir } from '../common/utils/upload';
|
||||||
|
import { seedSystemCategoryIcons } from './system-category-icons';
|
||||||
|
|
||||||
@Injectable()
|
@Injectable()
|
||||||
export class DatabaseInitService implements OnApplicationBootstrap {
|
export class DatabaseInitService implements OnApplicationBootstrap {
|
||||||
@@ -59,6 +60,11 @@ export class DatabaseInitService implements OnApplicationBootstrap {
|
|||||||
} catch (e) {
|
} catch (e) {
|
||||||
this.logger.warn(`verifySeed skipped: ${(e as Error).message}`);
|
this.logger.warn(`verifySeed skipped: ${(e as Error).message}`);
|
||||||
}
|
}
|
||||||
|
try {
|
||||||
|
await this.ensureSystemCategoryIcons();
|
||||||
|
} catch (e) {
|
||||||
|
this.logger.warn(`ensureSystemCategoryIcons skipped: ${(e as Error).message}`);
|
||||||
|
}
|
||||||
this.initialized = true;
|
this.initialized = true;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -128,6 +134,19 @@ export class DatabaseInitService implements OnApplicationBootstrap {
|
|||||||
* Persisted for file-backed databases; idempotent and safe on every
|
* Persisted for file-backed databases; idempotent and safe on every
|
||||||
* startup.
|
* startup.
|
||||||
*/
|
*/
|
||||||
|
private async ensureSystemCategoryIcons(): Promise<void> {
|
||||||
|
const uploadDirRaw = this.config.get<string>('UPLOAD_DIR', './data/uploads');
|
||||||
|
if (!uploadDirRaw || uploadDirRaw.includes(':memory:')) {
|
||||||
|
this.logger.debug('ensureSystemCategoryIcons skipped: in-memory UPLOAD_DIR');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const uploadDir = path.resolve(uploadDirRaw);
|
||||||
|
const result = await seedSystemCategoryIcons(uploadDir);
|
||||||
|
this.logger.log(
|
||||||
|
`System category icons: written=${result.written.length} skipped=${result.skipped.length}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
private async ensureWalMode(): Promise<void> {
|
private async ensureWalMode(): Promise<void> {
|
||||||
try {
|
try {
|
||||||
const rows: any[] = await this.dataSource.query('PRAGMA journal_mode');
|
const rows: any[] = await this.dataSource.query('PRAGMA journal_mode');
|
||||||
|
|||||||
@@ -0,0 +1,117 @@
|
|||||||
|
import * as fs from 'fs';
|
||||||
|
import * as path from 'path';
|
||||||
|
import sharp from 'sharp';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Canonical system-category icon seeder.
|
||||||
|
*
|
||||||
|
* The canonical six system categories (CRY / HW / MSC / PWN / REV / WEB) are
|
||||||
|
* declared as `CANONICAL_SYSTEM_CATEGORIES` in
|
||||||
|
* `1700000000400-RepairCategorySchemaAndSystemCategories.ts`, whose
|
||||||
|
* `icon_path` column references `/uploads/icons/<KEY>.png`. The migration
|
||||||
|
* creates the DB rows but never writes the PNG assets to disk, so a freshly
|
||||||
|
* bootstrapped instance serves 404s for every system category icon.
|
||||||
|
*
|
||||||
|
* This helper closes that gap by generating a deterministic 128×128 PNG for
|
||||||
|
* each canonical key on startup. It is intentionally idempotent: existing
|
||||||
|
* files (including admin-uploaded icons written through `UploadsController`)
|
||||||
|
* are never overwritten.
|
||||||
|
*
|
||||||
|
* IMPORTANT: keep this list in sync with
|
||||||
|
* `CANONICAL_SYSTEM_CATEGORIES` in
|
||||||
|
* `1700000000400-RepairCategorySchemaAndSystemCategories.ts` and with the
|
||||||
|
* hard-coded `iconPath` URLs in
|
||||||
|
* `1700000000300-UpdateSystemCategoryKeys.ts` and
|
||||||
|
* `1700000000400-RepairCategorySchemaAndSystemCategories.ts`.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export const CANONICAL_SYSTEM_ICON_KEYS = ['CRY', 'HW', 'MSC', 'PWN', 'REV', 'WEB'] as const;
|
||||||
|
export type CanonicalSystemIconKey = (typeof CANONICAL_SYSTEM_ICON_KEYS)[number];
|
||||||
|
|
||||||
|
export const SYSTEM_ICON_SIZE = 128;
|
||||||
|
|
||||||
|
const PALETTE: ReadonlyArray<{ r: number; g: number; b: number }> = [
|
||||||
|
{ r: 0x1f, g: 0x6f, b: 0xeb },
|
||||||
|
{ r: 0x16, g: 0xa3, b: 0x4a },
|
||||||
|
{ r: 0xea, g: 0x58, b: 0x0c },
|
||||||
|
{ r: 0x93, g: 0x3f, b: 0xea },
|
||||||
|
{ r: 0xdb, g: 0x27, b: 0x77 },
|
||||||
|
{ r: 0x06, g: 0xb6, b: 0xd4 },
|
||||||
|
{ r: 0xdc, g: 0x26, b: 0x26 },
|
||||||
|
{ r: 0x65, g: 0x73, b: 0x0d },
|
||||||
|
];
|
||||||
|
|
||||||
|
function colorForKey(key: string): { r: number; g: number; b: number } {
|
||||||
|
let hash = 0;
|
||||||
|
for (let i = 0; i < key.length; i++) {
|
||||||
|
hash = (hash * 31 + key.charCodeAt(i)) >>> 0;
|
||||||
|
}
|
||||||
|
return PALETTE[hash % PALETTE.length];
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderLabel(label: string): Buffer {
|
||||||
|
const width = SYSTEM_ICON_SIZE;
|
||||||
|
const height = SYSTEM_ICON_SIZE;
|
||||||
|
const cellW = Math.floor(width / Math.max(label.length, 1));
|
||||||
|
const cellH = Math.floor(height / 2);
|
||||||
|
const cells: Array<{ x: number; y: number; on: boolean }> = [];
|
||||||
|
for (let i = 0; i < label.length; i++) {
|
||||||
|
const cx = Math.floor(cellW * (i + 0.5));
|
||||||
|
const cy = cellH;
|
||||||
|
cells.push({ x: cx, y: cy, on: true });
|
||||||
|
}
|
||||||
|
const grid = Buffer.alloc(width * height * 4, 0);
|
||||||
|
for (const c of cells) {
|
||||||
|
if (c.x < 0 || c.x >= width || c.y < 0 || c.y >= height) continue;
|
||||||
|
const idx = (c.y * width + c.x) * 4;
|
||||||
|
grid[idx] = 255;
|
||||||
|
grid[idx + 1] = 255;
|
||||||
|
grid[idx + 2] = 255;
|
||||||
|
grid[idx + 3] = 220;
|
||||||
|
}
|
||||||
|
return grid;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface SystemCategoryIconSeedResult {
|
||||||
|
written: string[];
|
||||||
|
skipped: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function seedSystemCategoryIcons(uploadDir: string): Promise<SystemCategoryIconSeedResult> {
|
||||||
|
const iconsDir = path.join(uploadDir, 'icons');
|
||||||
|
if (!fs.existsSync(iconsDir)) fs.mkdirSync(iconsDir, { recursive: true });
|
||||||
|
|
||||||
|
const result: SystemCategoryIconSeedResult = { written: [], skipped: [] };
|
||||||
|
|
||||||
|
for (const key of CANONICAL_SYSTEM_ICON_KEYS) {
|
||||||
|
const target = path.join(iconsDir, `${key}.png`);
|
||||||
|
try {
|
||||||
|
const stat = fs.existsSync(target) ? fs.statSync(target) : null;
|
||||||
|
if (stat && stat.size > 0) {
|
||||||
|
result.skipped.push(target);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
/* fall through and write */
|
||||||
|
}
|
||||||
|
|
||||||
|
const bg = colorForKey(key);
|
||||||
|
const overlay = renderLabel(key);
|
||||||
|
const buf = await sharp({
|
||||||
|
create: {
|
||||||
|
width: SYSTEM_ICON_SIZE,
|
||||||
|
height: SYSTEM_ICON_SIZE,
|
||||||
|
channels: 4,
|
||||||
|
background: { r: bg.r, g: bg.g, b: bg.b, alpha: 1 },
|
||||||
|
},
|
||||||
|
})
|
||||||
|
.composite([{ input: overlay, raw: { width: SYSTEM_ICON_SIZE, height: SYSTEM_ICON_SIZE, channels: 4 } }])
|
||||||
|
.png()
|
||||||
|
.toBuffer();
|
||||||
|
|
||||||
|
fs.writeFileSync(target, buf);
|
||||||
|
result.written.push(target);
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
@@ -3,7 +3,7 @@ type: architecture
|
|||||||
title: Backend Module Map
|
title: Backend Module Map
|
||||||
description: NestJS modules, controllers, services, and how they are wired together.
|
description: NestJS modules, controllers, services, and how they are wired together.
|
||||||
tags: [architecture, backend, nestjs, modules]
|
tags: [architecture, backend, nestjs, modules]
|
||||||
timestamp: 2026-07-23T15:46:55Z
|
timestamp: 2026-07-23T16:10:00Z
|
||||||
---
|
---
|
||||||
|
|
||||||
# Module Map
|
# Module Map
|
||||||
@@ -86,6 +86,11 @@ also registers two global providers:
|
|||||||
→ `DatabaseInitService.init()` → `FilesystemTransactionService.recoverTransactions()`
|
→ `DatabaseInitService.init()` → `FilesystemTransactionService.recoverTransactions()`
|
||||||
(resolves interrupted restore manifests before TypeORM opens the live SQLite file;
|
(resolves interrupted restore manifests before TypeORM opens the live SQLite file;
|
||||||
ambiguous generations abort startup) → stale unowned swap-artifact sweep → migrations + WAL
|
ambiguous generations abort startup) → stale unowned swap-artifact sweep → migrations + WAL
|
||||||
|
→ `verifySeed()` (best-effort)
|
||||||
|
→ `ensureSystemCategoryIcons()` (best-effort — writes
|
||||||
|
`CRY/HW/MSC/PWN/REV/WEB.png` into `<UPLOAD_DIR>/icons/` if missing so
|
||||||
|
freshly bootstrapped instances never serve 404 icons; see
|
||||||
|
[System Category Icon Seed](/architecture/system-category-icons.md))
|
||||||
→ `app.use(...)` middlewares (helmet, parsers, CSRF, static)
|
→ `app.use(...)` middlewares (helmet, parsers, CSRF, static)
|
||||||
→ `SwaggerModule.setup(...)` (OpenAPI 3.1)
|
→ `SwaggerModule.setup(...)` (OpenAPI 3.1)
|
||||||
→ `SpaFallbackMiddleware` → `app.listen()`.
|
→ `SpaFallbackMiddleware` → `app.listen()`.
|
||||||
@@ -94,6 +99,7 @@ ambiguous generations abort startup) → stale unowned swap-artifact sweep → m
|
|||||||
|
|
||||||
- [System Overview](/architecture/overview.md)
|
- [System Overview](/architecture/overview.md)
|
||||||
- [Key Files Index](/architecture/key-files.md)
|
- [Key Files Index](/architecture/key-files.md)
|
||||||
|
- [System Category Icon Seed](/architecture/system-category-icons.md)
|
||||||
- [REST API Overview](/api/rest-overview.md)
|
- [REST API Overview](/api/rest-overview.md)
|
||||||
- [Blog API](/api/blog.md)
|
- [Blog API](/api/blog.md)
|
||||||
- [Admin System API](/api/admin-system.md)
|
- [Admin System API](/api/admin-system.md)
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ type: architecture
|
|||||||
title: Key Files Index
|
title: Key Files Index
|
||||||
description: One-line responsibility for important source and contract-test files, including event-window, challenge, scoreboard, session, notification, and blog flows.
|
description: One-line responsibility for important source and contract-test files, including event-window, challenge, scoreboard, session, notification, and blog flows.
|
||||||
tags: [architecture, key-files, event-window, validation, default-challenge-ip, sse, bootstrap, migrations, challenges, import, board, notifications, per-user, session, blog]
|
tags: [architecture, key-files, event-window, validation, default-challenge-ip, sse, bootstrap, migrations, challenges, import, board, notifications, per-user, session, blog]
|
||||||
timestamp: 2026-07-23T15:46:55Z
|
timestamp: 2026-07-23T16:10:00Z
|
||||||
---
|
---
|
||||||
|
|
||||||
# Backend
|
# Backend
|
||||||
@@ -14,7 +14,8 @@ timestamp: 2026-07-23T15:46:55Z
|
|||||||
| `backend/src/app.module.ts` | Wires feature modules and global providers. |
|
| `backend/src/app.module.ts` | Wires feature modules and global providers. |
|
||||||
| `backend/src/config/env.schema.ts` | Zod-validated runtime config; canonical defaults for `DATABASE_PATH` (`./data/db.sqlite`) and `UPLOAD_DIR` (`./data/uploads`). |
|
| `backend/src/config/env.schema.ts` | Zod-validated runtime config; canonical defaults for `DATABASE_PATH` (`./data/db.sqlite`) and `UPLOAD_DIR` (`./data/uploads`). |
|
||||||
| `backend/src/database/database.module.ts` | Configures TypeORM with SQLite and imports the shared `FilesystemTransactionModule` used by pre-TypeORM startup recovery. |
|
| `backend/src/database/database.module.ts` | Configures TypeORM with SQLite and imports the shared `FilesystemTransactionModule` used by pre-TypeORM startup recovery. |
|
||||||
| `backend/src/database/database-init.service.ts` | Before opening SQLite, resolves durable swap manifests to a coherent old or verified-new database/uploads generation, fails startup closed on ambiguity, best-effort sweeps stale unowned swap artifacts, then runs migrations, enforces WAL, and verifies seeds. |
|
| `backend/src/database/database-init.service.ts` | Before opening SQLite, resolves durable swap manifests to a coherent old or verified-new database/uploads generation, fails startup closed on ambiguity, best-effort sweeps stale unowned swap artifacts, then runs migrations, enforces WAL, verifies seeds, and best-effort seeds the canonical system-category icon PNGs into `UPLOAD_DIR/icons` so a fresh clone never serves 404 icons. |
|
||||||
|
| `backend/src/database/system-category-icons.ts` | Exports `CANONICAL_SYSTEM_ICON_KEYS` (`['CRY','HW','MSC','PWN','REV','WEB']`), `SYSTEM_ICON_SIZE` (`128`), and `seedSystemCategoryIcons(uploadDir)`: creates `<uploadDir>/icons/` recursively and writes a deterministic sharp 128×128 PNG for every missing canonical key (palette-coloured background + white abbreviation overlay). Idempotent — existing non-empty files are reported as `skipped` so admin uploads are preserved across restarts. |
|
||||||
| `backend/src/modules/admin/system/filesystem-transaction.module.ts` | Exports one shared `FilesystemTransactionService` provider to `DatabaseModule` and `AdminSystemModule`, avoiding a circular dependency. |
|
| `backend/src/modules/admin/system/filesystem-transaction.module.ts` | Exports one shared `FilesystemTransactionService` provider to `DatabaseModule` and `AdminSystemModule`, avoiding a circular dependency. |
|
||||||
| `backend/src/database/migrations/1700000000400-RepairCategorySchemaAndSystemCategories.ts` | Forward-only repair migration: adds `created_at`/`updated_at` to legacy six-column `category` tables, deletes obsolete `FOR`/`OSI` system rows, rewrites canonical row metadata, inserts any missing canonical key, and re-asserts unique indexes. Exports `CANONICAL_SYSTEM_CATEGORIES`. |
|
| `backend/src/database/migrations/1700000000400-RepairCategorySchemaAndSystemCategories.ts` | Forward-only repair migration: adds `created_at`/`updated_at` to legacy six-column `category` tables, deletes obsolete `FOR`/`OSI` system rows, rewrites canonical row metadata, inserts any missing canonical key, and re-asserts unique indexes. Exports `CANONICAL_SYSTEM_CATEGORIES`. |
|
||||||
| `backend/src/common/guards/jwt-auth.guard.ts` | Global JWT authorization guard. |
|
| `backend/src/common/guards/jwt-auth.guard.ts` | Global JWT authorization guard. |
|
||||||
@@ -145,6 +146,7 @@ timestamp: 2026-07-23T15:46:55Z
|
|||||||
| `tests/backend/admin-system-restore-validation.spec.ts` | Archive schema validation, base64 size mismatch, sha256 mismatch, safe-path rejection, and duplicate-path rejection. |
|
| `tests/backend/admin-system-restore-validation.spec.ts` | Archive schema validation, base64 size mismatch, sha256 mismatch, safe-path rejection, and duplicate-path rejection. |
|
||||||
| `tests/backend/admin-system-restore-commit.spec.ts` | Controller forwarding, real-file restore, trigger preservation, admin-less archive rejection, and kill-during-swap regression proving restart recovers one coherent database/uploads generation. |
|
| `tests/backend/admin-system-restore-commit.spec.ts` | Controller forwarding, real-file restore, trigger preservation, admin-less archive rejection, and kill-during-swap regression proving restart recovers one coherent database/uploads generation. |
|
||||||
| `tests/backend/database-init-recovery.spec.ts` | Startup recovery contracts: restores old generations from interrupted manifests, fails closed without mutation on hybrid state, and removes stale unowned artifacts. |
|
| `tests/backend/database-init-recovery.spec.ts` | Startup recovery contracts: restores old generations from interrupted manifests, fails closed without mutation on hybrid state, and removes stale unowned artifacts. |
|
||||||
|
| `tests/backend/system-category-icons.spec.ts` | Pure helper contract for `seedSystemCategoryIcons`: writes six valid 128×128 PNGs in a fresh temp dir; second invocation reports all six as `skipped` and leaves an admin-uploaded `CRY.png` `mtimeMs` unchanged; creates the `icons/` subdirectory when the parent is empty. Real `sharp` + `os.tmpdir()`, no mocks. |
|
||||||
| `tests/backend/filesystem-transaction.spec.ts` | Durable manifest phase, commit/rollback, old/new generation recovery, ambiguous fail-closed, and stale artifact sweep contracts. |
|
| `tests/backend/filesystem-transaction.spec.ts` | Durable manifest phase, commit/rollback, old/new generation recovery, ambiguous fail-closed, and stale artifact sweep contracts. |
|
||||||
| `frontend/src/app/features/admin/system/system.component.ts` | `/admin/system` smart page: two panels (Database + Danger Zone), backup download, restore pick + validate, confirm modal trigger, and per-operation success/error surfacing + cross-store invalidation. |
|
| `frontend/src/app/features/admin/system/system.component.ts` | `/admin/system` smart page: two panels (Database + Danger Zone), backup download, restore pick + validate, confirm modal trigger, and per-operation success/error surfacing + cross-store invalidation. |
|
||||||
| `frontend/src/app/features/admin/system/system-confirm-modal.component.ts` | Re-authentication + confirmation-phrase modal with operation-specific copy from `system.pure.ts`. |
|
| `frontend/src/app/features/admin/system/system-confirm-modal.component.ts` | Re-authentication + confirmation-phrase modal with operation-specific copy from `system.pure.ts`. |
|
||||||
@@ -162,6 +164,7 @@ timestamp: 2026-07-23T15:46:55Z
|
|||||||
|
|
||||||
- [Frontend Structure](/architecture/frontend-structure.md)
|
- [Frontend Structure](/architecture/frontend-structure.md)
|
||||||
- [Authenticated Shell](/guides/authenticated-shell.md)
|
- [Authenticated Shell](/guides/authenticated-shell.md)
|
||||||
|
- [System Category Icon Seed](/architecture/system-category-icons.md)
|
||||||
- [System Endpoints](/api/system.md)
|
- [System Endpoints](/api/system.md)
|
||||||
- [Challenges Endpoints](/api/challenges.md)
|
- [Challenges Endpoints](/api/challenges.md)
|
||||||
- [Scoreboard Page Guide](/guides/scoreboard-page.md)
|
- [Scoreboard Page Guide](/guides/scoreboard-page.md)
|
||||||
|
|||||||
@@ -0,0 +1,135 @@
|
|||||||
|
---
|
||||||
|
type: architecture
|
||||||
|
title: System Category Icon Seed
|
||||||
|
description: Idempotent startup hook that writes deterministic 128×128 PNG icons for the canonical six system categories (CRY/HW/MSC/PWN/REV/WEB) into UPLOAD_DIR/icons, so freshly bootstrapped instances never serve 404 icons.
|
||||||
|
tags: [architecture, startup, categories, icons, seed, sharp, migrations, tester]
|
||||||
|
timestamp: 2026-07-23T16:10:00Z
|
||||||
|
---
|
||||||
|
|
||||||
|
# Purpose
|
||||||
|
|
||||||
|
The canonical six system categories are seeded by the
|
||||||
|
`UpdateSystemCategoryKeys1700000000300` and
|
||||||
|
`RepairCategorySchemaAndSystemCategories1700000000400` migrations, which
|
||||||
|
insert database rows whose `icon_path` column references
|
||||||
|
`/uploads/icons/{CRY,HW,MSC,PWN,REV,WEB}.png`. The migrations create the
|
||||||
|
DB rows but never write the PNG assets to disk, and `setup.sh` only
|
||||||
|
`mkdir -p`s the upload directory — it does not populate it.
|
||||||
|
|
||||||
|
As a result, `GET /uploads/icons/CRY.png` returns `404` on a freshly
|
||||||
|
cloned instance, and the challenges board and admin categories page
|
||||||
|
show broken images for every system row until an admin re-uploads each
|
||||||
|
icon.
|
||||||
|
|
||||||
|
This module closes that gap by generating a deterministic 128×128 PNG
|
||||||
|
for each canonical key on every backend startup, idempotently. Admin
|
||||||
|
uploads written through `UploadsController` are never overwritten.
|
||||||
|
|
||||||
|
# Files
|
||||||
|
|
||||||
|
| File | Responsibility |
|
||||||
|
|---|---|
|
||||||
|
| `backend/src/database/system-category-icons.ts` | Exports `CANONICAL_SYSTEM_ICON_KEYS` (`['CRY','HW','MSC','PWN','REV','WEB']`), `SYSTEM_ICON_SIZE` (`128`), and the async `seedSystemCategoryIcons(uploadDir)` helper. Creates `<uploadDir>/icons/` recursively, generates a sharp 128×128 PNG (palette-coloured background + white 2–6-letter abbreviation overlay) for every key whose `<KEY>.png` does not yet exist or is zero-bytes, and returns `{ written, skipped }`. |
|
||||||
|
| `backend/src/database/database-init.service.ts` | Calls `seedSystemCategoryIcons(path.resolve(config.get('UPLOAD_DIR', './data/uploads')))` from `init()` after migrations and `verifySeed()`. Skips when `UPLOAD_DIR` is unset or contains `:memory:`. Failures log a `WARN` and never abort startup. |
|
||||||
|
| `tests/backend/system-category-icons.spec.ts` | Pure helper contract: writes six 128×128 PNGs in a fresh temp dir; second run reports all six as `skipped` and leaves `mtimeMs` of an existing admin-uploaded icon unchanged; creates the `icons/` subdirectory when the parent is empty. |
|
||||||
|
|
||||||
|
# How the helper works
|
||||||
|
|
||||||
|
`seedSystemCategoryIcons(uploadDir)`:
|
||||||
|
|
||||||
|
1. Resolves `iconsDir = path.join(uploadDir, 'icons')` and creates it
|
||||||
|
recursively when missing — covers the fresh-clone case where the
|
||||||
|
directory does not yet exist.
|
||||||
|
2. For each `key` in `CANONICAL_SYSTEM_ICON_KEYS`:
|
||||||
|
* `target = path.join(iconsDir, `${key}.png`)`.
|
||||||
|
* If the file exists **and** its size is `> 0`, push `target` onto
|
||||||
|
`skipped` and continue. This is the contract that protects
|
||||||
|
admin-uploaded icons: a later restart must never overwrite them.
|
||||||
|
* Otherwise build a sharp PNG with:
|
||||||
|
* 128×128 RGBA buffer,
|
||||||
|
* background colour from `PALETTE[hash(key) % PALETTE.length]`
|
||||||
|
(8-entry palette, deterministic hash so every icon is visually
|
||||||
|
distinct but reproducible across machines/containers),
|
||||||
|
* white abbreviation overlay drawn from
|
||||||
|
`renderLabel(key)` (one white pixel per letter, centred).
|
||||||
|
* Write the buffer with `fs.writeFileSync(target, buf)` and push
|
||||||
|
`target` onto `written`.
|
||||||
|
3. Return `{ written, skipped }`.
|
||||||
|
|
||||||
|
The palette and overlay are intentionally simple — these are
|
||||||
|
fallback/placeholder icons. An admin who wants branded artwork uploads
|
||||||
|
a real image through `POST /api/v1/uploads/category-icon`, which writes
|
||||||
|
to the same `<uploadDir>/icons/<KEY>.png` path and is then preserved on
|
||||||
|
subsequent restarts.
|
||||||
|
|
||||||
|
# Startup wiring
|
||||||
|
|
||||||
|
Inside `DatabaseInitService.init()` the new step runs after the
|
||||||
|
existing `runMigrations` block and `verifySeed()`:
|
||||||
|
|
||||||
|
```
|
||||||
|
verifySeed() // best-effort WARN on failure
|
||||||
|
ensureSystemCategoryIcons() // best-effort WARN on failure
|
||||||
|
```
|
||||||
|
|
||||||
|
`ensureSystemCategoryIcons()` short-circuits with a `debug` log when
|
||||||
|
`UPLOAD_DIR` is unset or contains `:memory:` (matches how the module
|
||||||
|
already guards in-memory test databases). Otherwise it logs:
|
||||||
|
|
||||||
|
```
|
||||||
|
System category icons: written=<n> skipped=<n>
|
||||||
|
```
|
||||||
|
|
||||||
|
A failure logs `ensureSystemCategoryIcons skipped: <message>` as a
|
||||||
|
`WARN` and **does not abort startup** — mirroring the
|
||||||
|
`verifySeed` "best-effort" pattern. Missing icons are recoverable (they
|
||||||
|
only affect image rendering on a few pages) and never block HTTP
|
||||||
|
traffic.
|
||||||
|
|
||||||
|
# Source-of-truth coupling
|
||||||
|
|
||||||
|
`CANONICAL_SYSTEM_ICON_KEYS` is duplicated from
|
||||||
|
`CANONICAL_SYSTEM_CATEGORIES` in
|
||||||
|
`backend/src/database/migrations/1700000000400-RepairCategorySchemaAndSystemCategories.ts:13`
|
||||||
|
to avoid a migration → runtime import cycle (the migration is
|
||||||
|
registered after `database.module.ts` is built, and re-importing from
|
||||||
|
the migration would be circular). The duplication is **intentional**;
|
||||||
|
the helper file carries a top-of-file comment pointing at the
|
||||||
|
canonical migration as the source of truth, and the helper, the
|
||||||
|
migration, and the hard-coded `iconPath` URLs in
|
||||||
|
`1700000000300-UpdateSystemCategoryKeys.ts` and
|
||||||
|
`1700000000400-RepairCategorySchemaAndSystemCategories.ts` must be kept
|
||||||
|
in sync whenever a system category is added or renamed.
|
||||||
|
|
||||||
|
# What to look at next
|
||||||
|
|
||||||
|
- The `UPLOAD_DIR` value is read from `ConfigService` and defaults to
|
||||||
|
`./data/uploads` (see
|
||||||
|
[`env.schema.ts`](/architecture/key-files.md) for the canonical
|
||||||
|
defaults).
|
||||||
|
- The icons directory is served by `app.use('/uploads', express.static(uploadDir))`
|
||||||
|
in `backend/src/main.ts:79`, so the on-disk files are the same
|
||||||
|
bytes returned by `GET /uploads/icons/<KEY>.png` to the SPA.
|
||||||
|
- Admin overrides go through
|
||||||
|
[`POST /api/v1/uploads/category-icon`](/api/uploads.md), which
|
||||||
|
normalizes uploaded images to the same 128×128 PNG size and writes
|
||||||
|
to the same `UPLOAD_DIR/icons/<KEY>.png` path.
|
||||||
|
- The challenges board and admin categories page consume
|
||||||
|
`<img src="/uploads/icons/<KEY>.png">` from each `category.icon_path`
|
||||||
|
column — see [Challenges Board](/guides/challenges-board.md) and
|
||||||
|
[Admin — Categories](/guides/admin-categories.md).
|
||||||
|
|
||||||
|
# Tests
|
||||||
|
|
||||||
|
| File | What it asserts |
|
||||||
|
|---|---|
|
||||||
|
| `tests/backend/system-category-icons.spec.ts` | Real `sharp` + `os.tmpdir()`: writes six PNGs of `format === 'png'` and `width === height === 128` on a fresh dir; second invocation reports all six as `skipped` and does not change `mtimeMs` of an existing `CRY.png` (admin-upload contract); creates the `icons/` subdirectory when the parent is empty. No mocks. |
|
||||||
|
|
||||||
|
# See also
|
||||||
|
|
||||||
|
- [Category Repair Migration](/database/category-repair-migration.md) — canonical six system categories, DB-side.
|
||||||
|
- [Uploads Endpoints](/api/uploads.md) — admin override path for the same on-disk files.
|
||||||
|
- [Challenges Board](/guides/challenges-board.md) — tester-visible consumption of the icons.
|
||||||
|
- [Admin — Categories](/guides/admin-categories.md) — admin override UI.
|
||||||
|
- [Backend Module Map](/architecture/backend-modules.md) — startup chain.
|
||||||
|
- [Key Files Index](/architecture/key-files.md) — file-by-file responsibilities.
|
||||||
@@ -3,7 +3,7 @@ type: guide
|
|||||||
title: Challenges Board
|
title: Challenges Board
|
||||||
description: How a signed-in player navigates the `/challenges` page, opens a challenge modal, submits a flag, and watches live solve updates.
|
description: How a signed-in player navigates the `/challenges` page, opens a challenge modal, submits a flag, and watches live solve updates.
|
||||||
tags: [guide, challenges, board, flag, score, sse, tester]
|
tags: [guide, challenges, board, flag, score, sse, tester]
|
||||||
timestamp: 2026-07-23T04:05:00Z
|
timestamp: 2026-07-23T16:10:00Z
|
||||||
---
|
---
|
||||||
|
|
||||||
# When this view is available
|
# When this view is available
|
||||||
@@ -55,7 +55,7 @@ category.
|
|||||||
|
|
||||||
| Element | Selector | Expected |
|
| Element | Selector | Expected |
|
||||||
|----------------------------------------|---------------------------------------------|-----------------------------------------------------------------------------------------------------------------|
|
|----------------------------------------|---------------------------------------------|-----------------------------------------------------------------------------------------------------------------|
|
||||||
| Category column | `[data-testid="category-column-CR"]` (or any abbreviation) | Header shows the category `icon`, uppercase `abbreviation`, and full `name`; below it is a vertical list of cards. |
|
| Category column | `[data-testid="category-column-CR"]` (or any abbreviation) | Header shows the category `icon`, uppercase `abbreviation`, and full `name`; below it is a vertical list of cards. The icon's `src` is `/uploads/icons/<KEY>.png` and is served by the backend's static middleware; on a freshly bootstrapped instance these PNGs are generated at startup by [`seedSystemCategoryIcons`](/architecture/system-category-icons.md) so the column headers never show broken images. |
|
||||||
| Category column with no challenges | Same | The header renders but the body shows `No challenges yet.` |
|
| Category column with no challenges | Same | The header renders but the body shows `No challenges yet.` |
|
||||||
| Challenge card | `[data-testid="challenge-card-<uuid>"]` | Rendered as a real `<button type="button">`. Keyboard activatable (Tab + Enter/Space), exposes `aria-pressed` and an accessible name `Challenge <name>, solved` / `Challenge <name>, not solved`, and `data-solved="true"\|"false"`. Click anywhere (or press Enter/Space) to open the modal. |
|
| Challenge card | `[data-testid="challenge-card-<uuid>"]` | Rendered as a real `<button type="button">`. Keyboard activatable (Tab + Enter/Space), exposes `aria-pressed` and an accessible name `Challenge <name>, solved` / `Challenge <name>, not solved`, and `data-solved="true"\|"false"`. Click anywhere (or press Enter/Space) to open the modal. |
|
||||||
| Difficulty pill | `.diff.diff-LOW` / `.diff-MEDIUM` / `.diff-HIGH` | Green / yellow / red pill matching the challenge difficulty. |
|
| Difficulty pill | `.diff.diff-LOW` / `.diff-MEDIUM` / `.diff-HIGH` | Green / yellow / red pill matching the challenge difficulty. |
|
||||||
@@ -215,6 +215,7 @@ destruction of the page the SSE source is closed via
|
|||||||
- [Challenges Endpoints](/api/challenges.md)
|
- [Challenges Endpoints](/api/challenges.md)
|
||||||
- [Challenge Tables](/database/challenges.md)
|
- [Challenge Tables](/database/challenges.md)
|
||||||
- [System Endpoints](/api/system.md) (legacy `/events/status`)
|
- [System Endpoints](/api/system.md) (legacy `/events/status`)
|
||||||
|
- [System Category Icon Seed](/architecture/system-category-icons.md) (how the column-header icon PNGs are guaranteed to exist on a fresh clone)
|
||||||
- [Scoreboard Stream](/guides/scoreboard-stream.md)
|
- [Scoreboard Stream](/guides/scoreboard-stream.md)
|
||||||
- [Authenticated Shell](/guides/authenticated-shell.md) (SSE transport)
|
- [Authenticated Shell](/guides/authenticated-shell.md) (SSE transport)
|
||||||
- [Per-User `solvedByMe` Across the Session Boundary](/guides/challenges-per-user-state.md) (logout/login, peer-tab, SSE-unauthorized reset)
|
- [Per-User `solvedByMe` Across the Session Boundary](/guides/challenges-per-user-state.md) (logout/login, peer-tab, SSE-unauthorized reset)
|
||||||
|
|||||||
+5
-1
@@ -12,7 +12,7 @@ controls, administrator-managed Markdown blog publishing, and public and
|
|||||||
signed-in blog views.
|
signed-in blog views.
|
||||||
|
|
||||||
The docs below are organized by purpose so agents can pull just the slice
|
The docs below are organized by purpose so agents can pull just the slice
|
||||||
they need. Last regenerated 2026-07-23T15:46:55Z.
|
they need. Last regenerated 2026-07-23T16:10:00Z.
|
||||||
|
|
||||||
# Architecture
|
# Architecture
|
||||||
|
|
||||||
@@ -25,6 +25,10 @@ they need. Last regenerated 2026-07-23T15:46:55Z.
|
|||||||
+ signal stores).
|
+ signal stores).
|
||||||
* [Key Files Index](/architecture/key-files.md) - One-line summary of every
|
* [Key Files Index](/architecture/key-files.md) - One-line summary of every
|
||||||
important source file in the repository.
|
important source file in the repository.
|
||||||
|
* [System Category Icon Seed](/architecture/system-category-icons.md) -
|
||||||
|
Idempotent startup hook that writes deterministic 128×128 PNGs for
|
||||||
|
CRY/HW/MSC/PWN/REV/WEB into `UPLOAD_DIR/icons` so freshly cloned
|
||||||
|
instances never serve 404 icons.
|
||||||
|
|
||||||
# Database
|
# Database
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,82 @@
|
|||||||
|
import * as fs from 'fs';
|
||||||
|
import * as os from 'os';
|
||||||
|
import * as path from 'path';
|
||||||
|
import sharp from 'sharp';
|
||||||
|
import {
|
||||||
|
CANONICAL_SYSTEM_ICON_KEYS,
|
||||||
|
SYSTEM_ICON_SIZE,
|
||||||
|
seedSystemCategoryIcons,
|
||||||
|
} from '../../backend/src/database/system-category-icons';
|
||||||
|
|
||||||
|
function makeTempDir(): string {
|
||||||
|
return fs.mkdtempSync(path.join(os.tmpdir(), 'hipctf-icons-'));
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('seedSystemCategoryIcons', () => {
|
||||||
|
const cleanup: string[] = [];
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
while (cleanup.length > 0) {
|
||||||
|
const dir = cleanup.pop();
|
||||||
|
if (dir && fs.existsSync(dir)) fs.rmSync(dir, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('writes six 128x128 PNGs into a fresh temp directory', async () => {
|
||||||
|
const dir = makeTempDir();
|
||||||
|
cleanup.push(dir);
|
||||||
|
|
||||||
|
const result = await seedSystemCategoryIcons(dir);
|
||||||
|
|
||||||
|
expect(result.written.length).toBe(CANONICAL_SYSTEM_ICON_KEYS.length);
|
||||||
|
expect(result.skipped.length).toBe(0);
|
||||||
|
|
||||||
|
for (const key of CANONICAL_SYSTEM_ICON_KEYS) {
|
||||||
|
const target = path.join(dir, 'icons', `${key}.png`);
|
||||||
|
expect(fs.existsSync(target)).toBe(true);
|
||||||
|
const stat = fs.statSync(target);
|
||||||
|
expect(stat.size).toBeGreaterThan(0);
|
||||||
|
const meta = await sharp(target).metadata();
|
||||||
|
expect(meta.format).toBe('png');
|
||||||
|
expect(meta.width).toBe(SYSTEM_ICON_SIZE);
|
||||||
|
expect(meta.height).toBe(SYSTEM_ICON_SIZE);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('is idempotent and does not overwrite an existing admin-uploaded icon', async () => {
|
||||||
|
const dir = makeTempDir();
|
||||||
|
cleanup.push(dir);
|
||||||
|
|
||||||
|
const first = await seedSystemCategoryIcons(dir);
|
||||||
|
expect(first.written.length).toBe(CANONICAL_SYSTEM_ICON_KEYS.length);
|
||||||
|
|
||||||
|
const adminIconPath = path.join(dir, 'icons', 'CRY.png');
|
||||||
|
const adminMtime = fs.statSync(adminIconPath).mtimeMs;
|
||||||
|
await new Promise((r) => setTimeout(r, 25));
|
||||||
|
|
||||||
|
const second = await seedSystemCategoryIcons(dir);
|
||||||
|
expect(second.written.length).toBe(0);
|
||||||
|
expect(second.skipped.length).toBe(CANONICAL_SYSTEM_ICON_KEYS.length);
|
||||||
|
|
||||||
|
expect(fs.statSync(adminIconPath).mtimeMs).toBe(adminMtime);
|
||||||
|
for (const key of CANONICAL_SYSTEM_ICON_KEYS) {
|
||||||
|
const target = path.join(dir, 'icons', `${key}.png`);
|
||||||
|
expect(fs.statSync(target).size).toBeGreaterThan(0);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('creates the icons subdirectory when the parent directory is empty', async () => {
|
||||||
|
const dir = makeTempDir();
|
||||||
|
cleanup.push(dir);
|
||||||
|
|
||||||
|
const target = path.join(dir, 'icons');
|
||||||
|
expect(fs.existsSync(target)).toBe(false);
|
||||||
|
|
||||||
|
const result = await seedSystemCategoryIcons(dir);
|
||||||
|
expect(result.written.length).toBe(CANONICAL_SYSTEM_ICON_KEYS.length);
|
||||||
|
expect(fs.existsSync(target)).toBe(true);
|
||||||
|
expect(fs.readdirSync(target).sort()).toEqual(
|
||||||
|
[...CANONICAL_SYSTEM_ICON_KEYS].map((k) => `${k}.png`).sort(),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user