---
type: database
title: Category Repair Migration (1700000000400)
description: Forward-only migration that repairs legacy `category` schemas (adds timestamps) and reconciles system rows to the canonical CRY/HW/MSC/PWN/REV/WEB set.
tags: [database, migration, category, repair, system-categories]
timestamp: 2026-07-22T16:44:54Z
---

# Purpose

Some persistent HIPCTF databases (default path
[`./data/db.sqlite`](/database/schema.md)) were created before
the canonical six system categories existed. They contain a legacy
six-column `category` table (no `created_at` / `updated_at`) and
system rows whose `system_key` / `abbreviation` no longer match the
canonical set (e.g. `FOR` for Forensics, `OSI` for OSINT, missing
`HW` / `REV`).

When the backend starts against such a database, `CategoryEntity`
fails to `SELECT c.*` because the timestamp columns are absent, and
`GET /api/v1/admin/categories` returns 500. Migrations that may
already be recorded in those databases cannot be edited or relied on
to re-run, so a new forward-only migration is required.

# File

`backend/src/database/migrations/1700000000400-RepairCategorySchemaAndSystemCategories.ts`

Registered last in `backend/src/database/database.module.ts` so it
runs after `UpdateSystemCategoryKeys1700000000300`.

# Canonical system categories

Exported as `CANONICAL_SYSTEM_CATEGORIES` (typed
`ReadonlyArray<CanonicalSystemCategory>`) so the migration and any
future seed consumer share one source of truth:

| `system_key` | `name`                  | `abbreviation` | `description`                | `icon_path`                  |
|--------------|-------------------------|----------------|------------------------------|------------------------------|
| `CRY`        | Cryptography            | `CRY`          | Cryptographic challenges     | `/uploads/icons/CRY.png`     |
| `HW`         | Hardware                | `HW`           | Hardware challenges          | `/uploads/icons/HW.png`      |
| `MSC`        | Misc                    | `MSC`          | Miscellaneous challenges     | `/uploads/icons/MSC.png`     |
| `PWN`        | Pwn                     | `PWN`          | Binary exploitation          | `/uploads/icons/PWN.png`     |
| `REV`        | Reverse Engineering     | `REV`          | Reverse engineering challenges | `/uploads/icons/REV.png`   |
| `WEB`        | Web                     | `WEB`          | Web exploitation challenges  | `/uploads/icons/WEB.png`     |

# What the `up()` does

1. **`ensureCategoryTimestamps`** — reads `PRAGMA table_info("category")`.
   * If `created_at` is absent → `ALTER TABLE "category" ADD COLUMN "created_at" TEXT NOT NULL DEFAULT ''`.
   * If `updated_at` is absent → `ALTER TABLE "category" ADD COLUMN "updated_at" TEXT NOT NULL DEFAULT ''`.
   * `UPDATE "category" SET "created_at" = strftime(...)` and same for `updated_at` for any row still blank.
2. **`reconcileSystemCategories`** — inside the migration's
   transaction:
   * Loads all rows with `system_key IS NOT NULL`.
   * Deletes every system row whose `system_key` is **not** in the
     canonical set (e.g. legacy `FOR`, `OSI`).
   * For each canonical key:
     * If a row already exists with that `system_key` → updates
       `name`, `abbreviation`, `description`, `icon_path`, bumps
       timestamps.
     * Else if a row exists with the matching `abbreviation` (no
       `system_key` or wrong key) → adopts it as the canonical system
       row by setting `system_key` and metadata.
     * Else → `INSERT` a fresh row with a new UUID v4.
3. **`ensureCategoryIndexes`** —
   `CREATE UNIQUE INDEX IF NOT EXISTS "uq_category_system_key" ON "category"("system_key") WHERE "system_key" IS NOT NULL;`
   and `CREATE UNIQUE INDEX IF NOT EXISTS "uq_category_abbreviation" ON "category"("abbreviation");`.

# Idempotency

Running the migration twice (or restarting after it has been recorded
plus re-running it in tests) leaves exactly six system rows — one per
canonical key — with no duplicates. The reconciliation step only
*deletes* non-canonical system rows and only *updates / inserts*
canonical keys, so user-created categories (`system_key IS NULL`) and
challenges attached to them are never touched.

# `down()`

Conservative: only re-asserts the unique indexes. It does **not**
remove canonical rows, delete user categories, or drop the new
timestamp columns, so a rollback cannot break challenge references.

# Startup verification

`DatabaseInitService.verifySeed()` was tightened (no schema change)
so that, after every startup, it now:

* Counts total categories and total settings (informational).
* Selects `system_key, abbreviation FROM "category" WHERE system_key IS NOT NULL ORDER BY abbreviation`.
* Compares the set to the canonical `['CRY','HW','MSC','PWN','REV','WEB']`
  and logs a `WARN Canonical system categories mismatch: missing=[...] duplicateSystemRows=N`
  if any canonical key is missing or duplicated.

# Tests

| File | What it asserts |
|---|---|
| `tests/backend/category-repair-migration.spec.ts` | Builds the exact legacy six-column `category` schema with CRY/FOR/MSC/OSI/PWN/WEB rows, runs the migration on an isolated in-memory SQLite DB, then asserts: timestamps exist and are populated; a TypeORM repository list query no longer throws; `GET /admin/categories` returns exactly `[CRY, HW, MSC, PWN, REV, WEB]` in API sort order with icon + description metadata; running the repair twice still leaves six unique system rows. |
| `tests/backend/migrations.spec.ts` | Registers the new migration in the fresh-schema migration list and strengthens canonical system-category assertions to require exactly CRY/HW/MSC/PWN/REV/WEB once each with populated metadata. |
| `tests/backend/database-init.spec.ts` | Asserts that startup initialization yields the exact canonical set and remains duplicate-free across repeated initialization/startup behavior. |

# See also

* [Database Schema Overview](/database/schema.md) — full migration list and bootstrap behavior.
* [Challenge Tables](/database/challenges.md) — `category` schema, indexes, and provenance of `created_at` / `updated_at`.
* [Admin — Categories](/guides/admin-categories.md) — user-visible behavior that depends on the canonical six keys.