AI Implementation feature(893): Admin Area General Settings and Categories 1.11 (#33)
This commit was merged in pull request #33.
This commit is contained in:
@@ -0,0 +1,107 @@
|
||||
---
|
||||
type: database
|
||||
title: Category Repair Migration (1700000000400)
|
||||
description: Forward-only migration that repairs legacy `category` schemas (adds timestamps) and reconciles system rows to the canonical CRY/HW/MSC/PWN/REV/WEB set.
|
||||
tags: [database, migration, category, repair, system-categories]
|
||||
timestamp: 2026-07-22T16:44:54Z
|
||||
---
|
||||
|
||||
# Purpose
|
||||
|
||||
Some persistent `/data/hipctf/db.sqlite` databases were created before
|
||||
the canonical six system categories existed. They contain a legacy
|
||||
six-column `category` table (no `created_at` / `updated_at`) and
|
||||
system rows whose `system_key` / `abbreviation` no longer match the
|
||||
canonical set (e.g. `FOR` for Forensics, `OSI` for OSINT, missing
|
||||
`HW` / `REV`).
|
||||
|
||||
When the backend starts against such a database, `CategoryEntity`
|
||||
fails to `SELECT c.*` because the timestamp columns are absent, and
|
||||
`GET /api/v1/admin/categories` returns 500. Migrations that may
|
||||
already be recorded in those databases cannot be edited or relied on
|
||||
to re-run, so a new forward-only migration is required.
|
||||
|
||||
# File
|
||||
|
||||
`backend/src/database/migrations/1700000000400-RepairCategorySchemaAndSystemCategories.ts`
|
||||
|
||||
Registered last in `backend/src/database/database.module.ts` so it
|
||||
runs after `UpdateSystemCategoryKeys1700000000300`.
|
||||
|
||||
# Canonical system categories
|
||||
|
||||
Exported as `CANONICAL_SYSTEM_CATEGORIES` (typed
|
||||
`ReadonlyArray<CanonicalSystemCategory>`) so the migration and any
|
||||
future seed consumer share one source of truth:
|
||||
|
||||
| `system_key` | `name` | `abbreviation` | `description` | `icon_path` |
|
||||
|--------------|-------------------------|----------------|------------------------------|------------------------------|
|
||||
| `CRY` | Cryptography | `CRY` | Cryptographic challenges | `/uploads/icons/CRY.png` |
|
||||
| `HW` | Hardware | `HW` | Hardware challenges | `/uploads/icons/HW.png` |
|
||||
| `MSC` | Misc | `MSC` | Miscellaneous challenges | `/uploads/icons/MSC.png` |
|
||||
| `PWN` | Pwn | `PWN` | Binary exploitation | `/uploads/icons/PWN.png` |
|
||||
| `REV` | Reverse Engineering | `REV` | Reverse engineering challenges | `/uploads/icons/REV.png` |
|
||||
| `WEB` | Web | `WEB` | Web exploitation challenges | `/uploads/icons/WEB.png` |
|
||||
|
||||
# What the `up()` does
|
||||
|
||||
1. **`ensureCategoryTimestamps`** — reads `PRAGMA table_info("category")`.
|
||||
* If `created_at` is absent → `ALTER TABLE "category" ADD COLUMN "created_at" TEXT NOT NULL DEFAULT ''`.
|
||||
* If `updated_at` is absent → `ALTER TABLE "category" ADD COLUMN "updated_at" TEXT NOT NULL DEFAULT ''`.
|
||||
* `UPDATE "category" SET "created_at" = strftime(...)` and same for `updated_at` for any row still blank.
|
||||
2. **`reconcileSystemCategories`** — inside the migration's
|
||||
transaction:
|
||||
* Loads all rows with `system_key IS NOT NULL`.
|
||||
* Deletes every system row whose `system_key` is **not** in the
|
||||
canonical set (e.g. legacy `FOR`, `OSI`).
|
||||
* For each canonical key:
|
||||
* If a row already exists with that `system_key` → updates
|
||||
`name`, `abbreviation`, `description`, `icon_path`, bumps
|
||||
timestamps.
|
||||
* Else if a row exists with the matching `abbreviation` (no
|
||||
`system_key` or wrong key) → adopts it as the canonical system
|
||||
row by setting `system_key` and metadata.
|
||||
* Else → `INSERT` a fresh row with a new UUID v4.
|
||||
3. **`ensureCategoryIndexes`** —
|
||||
`CREATE UNIQUE INDEX IF NOT EXISTS "uq_category_system_key" ON "category"("system_key") WHERE "system_key" IS NOT NULL;`
|
||||
and `CREATE UNIQUE INDEX IF NOT EXISTS "uq_category_abbreviation" ON "category"("abbreviation");`.
|
||||
|
||||
# Idempotency
|
||||
|
||||
Running the migration twice (or restarting after it has been recorded
|
||||
plus re-running it in tests) leaves exactly six system rows — one per
|
||||
canonical key — with no duplicates. The reconciliation step only
|
||||
*deletes* non-canonical system rows and only *updates / inserts*
|
||||
canonical keys, so user-created categories (`system_key IS NULL`) and
|
||||
challenges attached to them are never touched.
|
||||
|
||||
# `down()`
|
||||
|
||||
Conservative: only re-asserts the unique indexes. It does **not**
|
||||
remove canonical rows, delete user categories, or drop the new
|
||||
timestamp columns, so a rollback cannot break challenge references.
|
||||
|
||||
# Startup verification
|
||||
|
||||
`DatabaseInitService.verifySeed()` was tightened (no schema change)
|
||||
so that, after every startup, it now:
|
||||
|
||||
* Counts total categories and total settings (informational).
|
||||
* Selects `system_key, abbreviation FROM "category" WHERE system_key IS NOT NULL ORDER BY abbreviation`.
|
||||
* Compares the set to the canonical `['CRY','HW','MSC','PWN','REV','WEB']`
|
||||
and logs a `WARN Canonical system categories mismatch: missing=[...] duplicateSystemRows=N`
|
||||
if any canonical key is missing or duplicated.
|
||||
|
||||
# Tests
|
||||
|
||||
| File | What it asserts |
|
||||
|---|---|
|
||||
| `tests/backend/category-repair-migration.spec.ts` | Builds the exact legacy six-column `category` schema with CRY/FOR/MSC/OSI/PWN/WEB rows, runs the migration on an isolated in-memory SQLite DB, then asserts: timestamps exist and are populated; a TypeORM repository list query no longer throws; `GET /admin/categories` returns exactly `[CRY, HW, MSC, PWN, REV, WEB]` in API sort order with icon + description metadata; running the repair twice still leaves six unique system rows. |
|
||||
| `tests/backend/migrations.spec.ts` | Registers the new migration in the fresh-schema migration list and strengthens canonical system-category assertions to require exactly CRY/HW/MSC/PWN/REV/WEB once each with populated metadata. |
|
||||
| `tests/backend/database-init.spec.ts` | Asserts that startup initialization yields the exact canonical set and remains duplicate-free across repeated initialization/startup behavior. |
|
||||
|
||||
# See also
|
||||
|
||||
* [Database Schema Overview](/database/schema.md) — full migration list and bootstrap behavior.
|
||||
* [Challenge Tables](/database/challenges.md) — `category` schema, indexes, and provenance of `created_at` / `updated_at`.
|
||||
* [Admin — Categories](/guides/admin-categories.md) — user-visible behavior that depends on the canonical six keys.
|
||||
@@ -3,7 +3,7 @@ type: database
|
||||
title: Challenge Tables
|
||||
description: category, challenge, challenge_file, and solve tables — how CTF challenges and scoring are stored.
|
||||
tags: [database, challenge, category, solve]
|
||||
timestamp: 2026-07-22T14:50:25Z
|
||||
timestamp: 2026-07-22T16:44:54Z
|
||||
---
|
||||
|
||||
# Tables
|
||||
@@ -25,14 +25,19 @@ System rows are seeded by the
|
||||
`UpdateSystemCategoryKeys1700000000300` migration so the canonical
|
||||
keys are `CRY` (Cryptography), `MSC` (Misc), `PWN` (Binary
|
||||
exploitation), `REV` (Reverse Engineering), `WEB` (Web), and `HW`
|
||||
(Hardware). The migration drops any legacy seeded rows that no
|
||||
longer match the canonical set before inserting the missing entries.
|
||||
(Hardware). A forward-only repair migration,
|
||||
`RepairCategorySchemaAndSystemCategories1700000000400`, runs after
|
||||
that one on existing databases that pre-date the canonical six keys
|
||||
(it adds `created_at`/`updated_at` if missing, deletes obsolete
|
||||
`FOR`/`OSI` system rows, rewrites name/abbreviation/description/icon
|
||||
metadata on canonical rows, inserts any missing canonical row, and
|
||||
guarantees unique indexes on `system_key` and `abbreviation`).
|
||||
User-created categories leave `system_key` as `NULL`.
|
||||
|
||||
The uniqueness on `abbreviation` is enforced both by application
|
||||
validation (uppercased on save, duplicates rejected) and by the
|
||||
`uq_category_abbreviation` index created in migration
|
||||
`1700000000200`.
|
||||
`1700000000200` (re-asserted by `1700000000400`).
|
||||
|
||||
## `challenge`
|
||||
|
||||
|
||||
+10
-5
@@ -3,7 +3,7 @@ type: database
|
||||
title: Database Schema Overview
|
||||
description: SQLite (better-sqlite3) schema for HIPCTF: tables, relationships, indexes, and WAL journal mode.
|
||||
tags: [database, sqlite, typeorm, schema]
|
||||
timestamp: 2026-07-22T14:50:25Z
|
||||
timestamp: 2026-07-22T16:44:54Z
|
||||
---
|
||||
|
||||
# Overview
|
||||
@@ -19,10 +19,15 @@ by a second migration
|
||||
(`backend/src/database/migrations/1700000000100-SeedSystemData.ts`).
|
||||
The initial migration declares every column used by the entities
|
||||
(including `category.created_at` and `category.updated_at`), so the
|
||||
later `AddCategoryTimestampsAndUniqueAbbrev1700000000200` and
|
||||
`UpdateSystemCategoryKeys1700000000300` migrations only patch legacy
|
||||
databases that pre-date those columns and reseed the canonical system
|
||||
rows.
|
||||
later `AddCategoryTimestampsAndUniqueAbbrev1700000000200`,
|
||||
`UpdateSystemCategoryKeys1700000000300`, and
|
||||
`RepairCategorySchemaAndSystemCategories1700000000400` migrations only
|
||||
patch legacy databases that pre-date those columns and reseed the
|
||||
canonical system rows. The `...0400` migration is idempotent and
|
||||
forward-only: it adds `created_at`/`updated_at` to legacy six-column
|
||||
`category` tables, deletes obsolete `FOR`/`OSI` system rows, rewrites
|
||||
metadata on canonical rows, inserts any missing canonical row, and
|
||||
re-asserts the unique indexes on `system_key` and `abbreviation`.
|
||||
|
||||
# Tables
|
||||
|
||||
|
||||
Reference in New Issue
Block a user