diff --git a/docs/api/admin-system.md b/docs/api/admin-system.md index 9276b31..3d80f5e 100644 --- a/docs/api/admin-system.md +++ b/docs/api/admin-system.md @@ -4,7 +4,7 @@ title: Admin System Endpoints description: Admin-only destructive System operations: backup download, restore validate/commit, re-authenticated confirmation tokens, reset-scores, and wipe-challenges. resource: backend/src/modules/admin/system/admin-system.controller.ts tags: [api, admin, system, backup, restore, danger-zone, confirmation-tokens] -timestamp: 2026-07-23T13:27:25Z +timestamp: 2026-07-23T15:46:55Z --- # Overview @@ -127,9 +127,15 @@ Flow: by cloning the live DB, then clearing every application table and inserting the archived rows in FK-safe order. Uploads are copied aside as `.restore-stage-` and the live - database + uploads are swapped atomically via - `FilesystemTransactionService.stageSwap()` (with a `restore-backup` - rollback handle). + database + uploads are swapped as one recoverable generation via + `FilesystemTransactionService.stageSwap(..., { stagingDir })` (with a + `restore-backup` rollback handle). Before moving either live path, the + service writes `//manifest.json`. + The manifest advances through `prepared`, `live-snapshotted`, + `replacements-installed`, and `verified`; commit removes rollback and + manifest artifacts. SQLite `-wal` and `-shm` sidecars travel with the + database rollback generation and stale sidecars are removed from the + replacement. The offline clone rebuild begins by capturing the exact SQL of the deployment-wide `trg_user_last_admin_update` and @@ -161,6 +167,25 @@ Flow: revoked via `AuthService.revokeAllRefreshSessions()` so the SPA is forced back to `/login`. +### Startup recovery contract + +Before TypeORM opens SQLite, `DatabaseInitService.init()` calls +`FilesystemTransactionService.recoverTransactions()` using +`SYSTEM_OP_STAGING_DIR` (default `/.system-staging`): + +| Manifest phase | Required complete generation | Startup action | +|---|---|---| +| `prepared` | Existing live paths | Keep the old generation and remove the transaction manifest. | +| `live-snapshotted` | Every rollback path exists and corresponding live path is absent | Restore all rollback paths, including tracked SQLite sidecars. | +| `replacements-installed` or `verified` | Every replacement live path and rollback path exists | Keep the complete new generation and remove rollback/staged artifacts. | +| Any ambiguous or mixed state | Neither generation is complete | Throw `SYSTEM_RESTORE_FAILED`, preserve the manifest and artifacts, and abort startup. | +| `committed` or `rolled-back` | Terminal | Remove the transaction directory. | + +After manifest recovery, startup best-effort removes stale unowned +`.rollback-*`, `.restore-stage-*`, and `.wipe-stage-*` artifacts near the +configured database/uploads paths. Fresh artifacts and paths referenced by a +manifest are retained. + Returns `{ ok: true, operation: 'restore-backup' }`. ## POST /confirmations @@ -301,7 +326,9 @@ curl -sS ... -d '{"operation":"reset-scores","token":""}' | Restore | `backend/src/modules/admin/system/restore.service.ts` | | Confirmation tokens | `backend/src/modules/admin/system/confirmation-token.service.ts` | | Danger zone | `backend/src/modules/admin/system/danger-zone.service.ts` | -| Filesystem swap | `backend/src/modules/admin/system/filesystem-transaction.service.ts` | +| Filesystem swap + recovery | `backend/src/modules/admin/system/filesystem-transaction.service.ts` | +| Shared filesystem module | `backend/src/modules/admin/system/filesystem-transaction.module.ts` | +| Startup recovery wiring | `backend/src/database/database-init.service.ts`, `backend/src/database/database.module.ts` | | Staging dir helper | `backend/src/common/utils/upload.ts` (`resolveSystemStagingDir`) | | Password re-auth / revoke | `backend/src/modules/auth/auth.service.ts` (`reauthenticateAdmin`, `revokeAllRefreshSessions`) | | Error codes | `backend/src/common/errors/error-codes.ts` | @@ -313,4 +340,4 @@ curl -sS ... -d '{"operation":"reset-scores","token":""}' | Front-end route | `frontend/src/app/app.routes.ts` (admin child `system`) | | Cross-store invalidation | `frontend/src/app/core/services/system-data-change.service.ts` | | Forced logout helper | `frontend/src/app/core/services/auth.service.ts` (`forceServerInvalidation`) | -| Tests | `tests/backend/admin-system-authorization.spec.ts`, `admin-system-backup.spec.ts`, `admin-system-confirmation-token.spec.ts`, `admin-system-danger.spec.ts`, `admin-system-restore-validation.spec.ts`, `admin-system-restore-commit.spec.ts`; `tests/frontend/admin-system.pure.spec.ts` | +| Tests | `tests/backend/admin-system-authorization.spec.ts`, `admin-system-backup.spec.ts`, `admin-system-confirmation-token.spec.ts`, `admin-system-danger.spec.ts`, `admin-system-restore-validation.spec.ts`, `admin-system-restore-commit.spec.ts`, `database-init-recovery.spec.ts`, `filesystem-transaction.spec.ts`; `tests/frontend/admin-system.pure.spec.ts` | diff --git a/docs/architecture/backend-modules.md b/docs/architecture/backend-modules.md index 0066c3e..80e42f7 100644 --- a/docs/architecture/backend-modules.md +++ b/docs/architecture/backend-modules.md @@ -3,7 +3,7 @@ type: architecture title: Backend Module Map description: NestJS modules, controllers, services, and how they are wired together. tags: [architecture, backend, nestjs, modules] -timestamp: 2026-07-23T10:12:24Z +timestamp: 2026-07-23T15:46:55Z --- # Module Map @@ -20,7 +20,8 @@ also registers two global providers: | Module | Path | Responsibility | |------------------|-----------------------------------------------|-------------------------------------------------------------------------------------------------| -| `DatabaseModule` | `backend/src/database/database.module.ts` | Configures TypeORM with `better-sqlite3`, registers entities, runs migrations, enforces WAL. | +| `DatabaseModule` | `backend/src/database/database.module.ts` | Configures TypeORM with `better-sqlite3`, registers entities, runs migrations, enforces WAL, and imports `FilesystemTransactionModule` so startup recovery runs before the database is opened. | +| `FilesystemTransactionModule` | `backend/src/modules/admin/system/filesystem-transaction.module.ts` | Provides the shared durable filesystem swap service to both `DatabaseModule` startup recovery and `AdminSystemModule` restore operations without a circular dependency. | | `CommonModule` | `backend/src/common/common.module.ts` | Provides shared services (CSRF middleware class, backoff, registration rate limit, SSE hub, theme loader, etc.). | | `SettingsModule` | `backend/src/modules/settings/settings.module.ts` | Exposes `SettingsService` (get/set/getAll over `setting` table). | | `AuthModule` | `backend/src/modules/auth/auth.module.ts` | `AuthController` (`/api/v1/auth/{register,login,refresh,logout,me,change-password,csrf}`) + `AuthService` (login/refresh/logout/register-first-admin + public `register` + `getMe` + `changePassword`). Imports `SettingsModule` so the public-register flow can read the `registrationsEnabled` flag, and `UsersModule` (forwardRef) for `UsersRankService`. | @@ -28,7 +29,7 @@ also registers two global providers: | `SetupModule` | `backend/src/modules/setup/setup.module.ts` | `SetupController` (`POST /api/v1/setup/create-admin`) + `SetupService`. Returns 409 `SYSTEM_INITIALIZED` once any admin exists; serializes concurrent first-admin attempts via an in-process promise chain. | | `SystemModule` | `backend/src/modules/system/system.module.ts` | `SystemController` (`/api/v1/bootstrap`, `/event/status`, SSE streams) + `SystemService`. | | `AdminModule` | `backend/src/modules/admin/admin.module.ts` | Five controllers (`AdminController` for `/api/v1/admin/users*`, `AdminGeneralController` for `/api/v1/admin/general/*`, `AdminCategoriesController` for `/api/v1/admin/categories*`, `AdminChallengesController` for `/api/v1/admin/challenges*`, `AdminSystemController` for `/api/v1/admin/system/*`) plus their services (`AdminService`, `AdminGeneralService`, `AdminCategoriesService`, `AdminChallengesService`, `ChallengeFilesService`, `BackupService`, `RestoreService`, `DangerZoneService`, `FilesystemTransactionService`, `ConfirmationTokenService`). Gated by `AdminGuard` + `@Roles('admin')`. Imports `TypeOrmModule.forFeature([UserEntity, CategoryEntity, ChallengeEntity, ChallengeFileEntity, SolveEntity])`, `AuthModule`, `UsersModule`, `SettingsModule`, `CommonModule` (for `SseHubService`, `ThemeLoaderService`), and `AdminSystemModule` (which adds `AdminOperationTokenEntity` + `AuthModule`). | -| `AdminSystemModule` | `backend/src/modules/admin/system/admin-system.module.ts` | Hosts the destructive System operations: `AdminSystemController` + `BackupService` + `RestoreService` + `DangerZoneService` + `FilesystemTransactionService` + `ConfirmationTokenService`. Imports `TypeOrmModule.forFeature([AdminOperationTokenEntity])` and `AuthModule` (for `reauthenticateAdmin`/`revokeAllRefreshSessions`). | +| `AdminSystemModule` | `backend/src/modules/admin/system/admin-system.module.ts` | Hosts the destructive System operations: `AdminSystemController` + `BackupService` + `RestoreService` + `DangerZoneService` + `ConfirmationTokenService`. Imports shared `FilesystemTransactionModule`, `TypeOrmModule.forFeature([AdminOperationTokenEntity])`, and `AuthModule` (for `reauthenticateAdmin`/`revokeAllRefreshSessions`). | | `UploadsModule` | `backend/src/modules/uploads/uploads.module.ts` | `UploadsController` (`/api/v1/uploads/*`). Admin-only multipart. | | `BlogModule` | `backend/src/modules/blog/blog.module.ts` | `BlogController`/`BlogService` expose published posts at `GET /api/v1/blog/posts`; `AdminBlogController`/`AdminBlogService` provide guarded CRUD at `/api/v1/admin/blog/posts` using `BlogPostEntity`. | | `ChallengesModule` | `backend/src/modules/challenges/challenges.module.ts` | `ChallengesController` (`/api/v1/challenges/{board,status,:id,:id/solves}`) + `ChallengesEventsController` (authenticated SSE `/api/v1/events`) + `ChallengesService` (board, detail, submit, scoring util). | @@ -82,7 +83,9 @@ also registers two global providers: # Startup chain `main.ts` → `AppModule` (loads `ConfigModule`, then all feature modules) -→ `DatabaseInitService.init()` (runs migrations + WAL) +→ `DatabaseInitService.init()` → `FilesystemTransactionService.recoverTransactions()` +(resolves interrupted restore manifests before TypeORM opens the live SQLite file; +ambiguous generations abort startup) → stale unowned swap-artifact sweep → migrations + WAL → `app.use(...)` middlewares (helmet, parsers, CSRF, static) → `SwaggerModule.setup(...)` (OpenAPI 3.1) → `SpaFallbackMiddleware` → `app.listen()`. diff --git a/docs/architecture/key-files.md b/docs/architecture/key-files.md index 62167e8..e4fa4bf 100644 --- a/docs/architecture/key-files.md +++ b/docs/architecture/key-files.md @@ -3,7 +3,7 @@ type: architecture title: Key Files Index description: One-line responsibility for important source and contract-test files, including event-window, challenge, scoreboard, session, notification, and blog flows. tags: [architecture, key-files, event-window, validation, default-challenge-ip, sse, bootstrap, migrations, challenges, import, board, notifications, per-user, session, blog] -timestamp: 2026-07-23T13:27:25Z +timestamp: 2026-07-23T15:46:55Z --- # Backend @@ -13,8 +13,9 @@ timestamp: 2026-07-23T13:27:25Z | `backend/src/main.ts` | Bootstraps Nest, middleware, OpenAPI, static assets, and SPA fallback. | | `backend/src/app.module.ts` | Wires feature modules and global providers. | | `backend/src/config/env.schema.ts` | Zod-validated runtime config; canonical defaults for `DATABASE_PATH` (`./data/db.sqlite`) and `UPLOAD_DIR` (`./data/uploads`). | -| `backend/src/database/database.module.ts` | Configures TypeORM with SQLite. | -| `backend/src/database/database-init.service.ts` | Initializes the database and runs migrations; `verifySeed()` also asserts the canonical six system-category keys and reports missing/duplicate rows. | +| `backend/src/database/database.module.ts` | Configures TypeORM with SQLite and imports the shared `FilesystemTransactionModule` used by pre-TypeORM startup recovery. | +| `backend/src/database/database-init.service.ts` | Before opening SQLite, resolves durable swap manifests to a coherent old or verified-new database/uploads generation, fails startup closed on ambiguity, best-effort sweeps stale unowned swap artifacts, then runs migrations, enforces WAL, and verifies seeds. | +| `backend/src/modules/admin/system/filesystem-transaction.module.ts` | Exports one shared `FilesystemTransactionService` provider to `DatabaseModule` and `AdminSystemModule`, avoiding a circular dependency. | | `backend/src/database/migrations/1700000000400-RepairCategorySchemaAndSystemCategories.ts` | Forward-only repair migration: adds `created_at`/`updated_at` to legacy six-column `category` tables, deletes obsolete `FOR`/`OSI` system rows, rewrites canonical row metadata, inserts any missing canonical key, and re-asserts unique indexes. Exports `CANONICAL_SYSTEM_CATEGORIES`. | | `backend/src/common/guards/jwt-auth.guard.ts` | Global JWT authorization guard. | | `backend/src/common/middleware/csrf.middleware.ts` | Double-submit CSRF protection. | @@ -124,11 +125,11 @@ timestamp: 2026-07-23T13:27:25Z | `tests/frontend/scoreboard.pure.spec.ts` | Pure helpers: `applyRankingSort` (competition-rank numbering), `parseSolveEventIntoRanking` (add-or-increment + sort), `dedupEventLogBySolveId`, `applySolveToGraph` (anchors + plateau + top-10 trim), `mutateMatrixFromSolve`, `stablePlayerColorIndex`, `formatSolveDateTime`. | | `tests/frontend/scoreboard.store.spec.ts` | Store lifecycle + live merge: `loadAll` aggregation + idempotency, SSE `solve` frame applying to all four tabs, `stop()` teardown, exponential reconnect. | | `backend/src/modules/admin/system/admin-system.controller.ts` | Mounts `/api/v1/admin/system/*` (backup download, restore validate/commit, confirmations, scores/reset, challenges/wipe); all handlers require `AdminGuard` + `@Roles('admin')` and the destructive ones also consume a confirmation token. | -| `backend/src/modules/admin/system/admin-system.module.ts` | Wires `AdminSystemController`, `BackupService`, `RestoreService`, `DangerZoneService`, `FilesystemTransactionService`, `ConfirmationTokenService`; imports `TypeOrmModule.forFeature([AdminOperationTokenEntity])` and `AuthModule`. | +| `backend/src/modules/admin/system/admin-system.module.ts` | Wires `AdminSystemController`, `BackupService`, `RestoreService`, `DangerZoneService`, and `ConfirmationTokenService`; imports the shared `FilesystemTransactionModule`, `TypeOrmModule.forFeature([AdminOperationTokenEntity])`, and `AuthModule`. | | `backend/src/modules/admin/system/backup.service.ts` | Builds the full backup JSON: every application table (`sqlite_master` discovery, excluding `admin_operation_token`/`migrations`/`sqlite_*`), with base64-encoded uploads walked recursively from `UPLOAD_DIR` (skipping `.staging`). Exposes `discoverTables()`, `getUploadDir()`, and `BackupService.stringify(doc)`. | -| `backend/src/modules/admin/system/restore.service.ts` | Two-phase restore: `stageArchive(rawText, userId)` validates the archive (zod), decodes uploads into `/.system-staging/restore-/`, and returns a summary; `commitRestore(stagingId)` clones the live DB, captures and temporarily drops the `trg_user_last_admin_update` / `trg_user_last_admin_delete` triggers in the offline candidate, clears every application table, re-inserts archived rows in FK-safe order, asserts the archive still contains at least one admin, re-creates the captured triggers, then atomically swaps the live DB + uploads via `FilesystemTransactionService.stageSwap({ name: 'restore-backup', ... })`. On any failure the swap is rolled back and `SYSTEM_RESTORE_ROLLED_BACK` is returned; the rebuild also best-effort re-creates any missing trigger before rethrowing so a rolled-back candidate never leaks missing the `LAST_ADMIN` safety net. | +| `backend/src/modules/admin/system/restore.service.ts` | Two-phase restore: validates/stages the archive, rebuilds an offline SQLite candidate while preserving the last-admin triggers, then swaps database + uploads as one durable manifest-backed generation; tracks SQLite `-wal`/`-shm` sidecars and reconnects TypeORM only after verification. | | `backend/src/modules/admin/system/danger-zone.service.ts` | `resetScores()` deletes every `solve` row transactionally; `wipeChallenges()` snapshots `/challenges` to a side directory, deletes every challenge inside a transaction, physically removes the live `-challenges` directory post-commit, and either deletes the snapshot or restores from it on failure. | -| `backend/src/modules/admin/system/filesystem-transaction.service.ts` | Generic file/directory swap primitive: `stageSwap({ pairs, hooks })` renames each live path to a rollback location, then renames the staged path into place (with copy+unlink cross-device fallback); `commit(handle)` removes rollback artifacts; `rollback(handle)` restores the original live paths. Provides static helpers `rmSafe`, `copyDirSync`, `ensureDir`. | +| `backend/src/modules/admin/system/filesystem-transaction.service.ts` | Crash-safe multi-path swap primitive. Persists per-transaction phase/path manifests under `SYSTEM_OP_STAGING_DIR`, moves configured sidecars with their generation, commits or rolls back idempotently, restores a complete old generation after an interrupted snapshot, finalizes a complete verified new generation, fails closed on mixed/ambiguous state, and sweeps stale unowned rollback/restore/wipe artifacts. | | `backend/src/modules/admin/system/confirmation-token.service.ts` | Issues SHA-256-hashed single-use tokens with `SYSTEM_OP_CONFIRM_TOKEN_TTL_MS` TTL; `consume()` runs in a transaction with conditional `WHERE consumedAt IS NULL` updates so only the first concurrent caller succeeds. `purgeExpired()` deletes expired and >24h-old consumed rows. | | `backend/src/modules/admin/system/dto/admin-system.dto.ts` | Re-exports `ADMIN_OPERATION_KINDS` and zod contracts for the re-auth body, danger confirm body, and restore-commit body; declares the backup format identifier + version constants. | | `backend/src/database/entities/admin-operation-token.entity.ts` | TypeORM entity and the canonical `ADMIN_OPERATION_KINDS` literal union (`restore-backup`, `reset-scores`, `wipe-challenges`). | @@ -142,7 +143,9 @@ timestamp: 2026-07-23T13:27:25Z | `tests/backend/admin-system-confirmation-token.spec.ts` | Issue/consume/expire/reuse/mismatch flow and TTL config. | | `tests/backend/admin-system-danger.spec.ts` | Reset-scores and wipe-challenges happy path + DB/file rollback. | | `tests/backend/admin-system-restore-validation.spec.ts` | Archive schema validation, base64 size mismatch, sha256 mismatch, safe-path rejection, and duplicate-path rejection. | -| `tests/backend/admin-system-restore-commit.spec.ts` | Controller forwards `stagingId` to `RestoreService.commitRestore`, plus a real-file service-level regression that restores a backup containing the same admin as the live system, asserts swapped table settings + uploads + trigger preservation, and verifies an admin-less archive is rejected with `SYSTEM_RESTORE_ROLLED_BACK`. | +| `tests/backend/admin-system-restore-commit.spec.ts` | Controller forwarding, real-file restore, trigger preservation, admin-less archive rejection, and kill-during-swap regression proving restart recovers one coherent database/uploads generation. | +| `tests/backend/database-init-recovery.spec.ts` | Startup recovery contracts: restores old generations from interrupted manifests, fails closed without mutation on hybrid state, and removes stale unowned artifacts. | +| `tests/backend/filesystem-transaction.spec.ts` | Durable manifest phase, commit/rollback, old/new generation recovery, ambiguous fail-closed, and stale artifact sweep contracts. | | `frontend/src/app/features/admin/system/system.component.ts` | `/admin/system` smart page: two panels (Database + Danger Zone), backup download, restore pick + validate, confirm modal trigger, and per-operation success/error surfacing + cross-store invalidation. | | `frontend/src/app/features/admin/system/system-confirm-modal.component.ts` | Re-authentication + confirmation-phrase modal with operation-specific copy from `system.pure.ts`. | | `frontend/src/app/features/admin/system/system.service.ts` | HTTP client for the six `/api/v1/admin/system/*` endpoints. | diff --git a/docs/guides/admin-system-page.md b/docs/guides/admin-system-page.md index e2758c6..573a00a 100644 --- a/docs/guides/admin-system-page.md +++ b/docs/guides/admin-system-page.md @@ -3,7 +3,7 @@ type: guide title: Admin — System (Backup, Restore, Danger Zone) description: How an admin navigates to /admin/system, creates a backup, stages and commits a restore, resets all scores, and wipes challenges — including the re-authentication confirmation, progress, and side effects. tags: [guide, admin, system, backup, restore, danger-zone, confirmation] -timestamp: 2026-07-23T12:04:50Z +timestamp: 2026-07-23T15:46:55Z --- # Navigation @@ -157,7 +157,13 @@ messages. Notable mappings: password, type `RESTORE BACKUP`, click **Confirm**. 6. The modal closes, the page shows a brief *Restore complete. Logging out…* toast, and the SPA navigates to `/login`. -7. Sign in again with the same admin credentials. The data on the +7. If the server process is terminated during the database/uploads swap, + restart the server. Before accepting requests, startup recovery uses the + durable restore manifest to restore the complete old generation or finalize + the complete verified new generation; it never intentionally boots with a + mixed database/uploads generation. An ambiguous incomplete generation fails + startup closed and preserves the manifest and artifacts for diagnosis. +8. Sign in again with the same admin credentials. The data on the target environment now matches the source backup. ## Manual reset happy path diff --git a/docs/index.md b/docs/index.md index 444ae6a..eabb25b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -12,7 +12,7 @@ controls, administrator-managed Markdown blog publishing, and public and signed-in blog views. The docs below are organized by purpose so agents can pull just the slice -they need. Last regenerated 2026-07-23T13:27:25Z. +they need. Last regenerated 2026-07-23T15:46:55Z. # Architecture