feat: Data and run

This commit is contained in:
OpenVelo Agent
2026-07-22 19:56:31 +00:00
parent 3c146a92f9
commit 74e689f25f
17 changed files with 169 additions and 46 deletions
+1
View File
@@ -6,6 +6,7 @@ dist/
!.env.example
.DS_Store
coverage/
data/
backend/data/
backend/themes/.cache/
frontend/.angular/
-35
View File
@@ -1,35 +0,0 @@
# Implementation Plan: Admin Area General Settings and Categories 1.16
## 1. Architectural Reconnaissance
- **Codebase style & conventions:** TypeScript monorepo with a NestJS 10 REST backend and Angular 17 standalone components. Backend controllers currently delegate category persistence to TypeORM-backed services and validate JSON DTOs with Zod pipes; uploads use Nest `FileInterceptor`, in-memory Multer buffers, and Sharp. Frontend admin pages use standalone OnPush components, signals, typed reactive forms, and an `AdminService` wrapper around typed `HttpClient` calls converted to promises with `firstValueFrom`.
- **Data Layer:** SQLite through TypeORM. Category abbreviations are stored uniquely, are normalized to uppercase by `AdminCategoriesService`, and list queries order by `LOWER(abbreviation)`. No schema migration is needed because `UpdateCategorySchema`, the entity, and service update path already support abbreviation changes for non-system categories.
- **Test Framework & Structure:** Jest 29 with `ts-jest`, split into root-level dedicated `tests/backend` and `tests/frontend` projects and runnable together with root `npm test`. Backend endpoint tests use Nest's testing module, Supertest, an in-memory SQLite database, and an isolated upload directory; frontend tests favor exported pure component helpers rather than browser or visual checks.
- **Required Tools & Dependencies:** No new system tools, global CLIs, or package dependencies are required. Sharp is already installed in both the backend and root test dependencies and can validate/decode and normalize uploaded images. Existing `setup.sh` already creates persistent `/data/hipctf/uploads`, installs dependencies, rebuilds the native SQLite binding, and builds both workspaces, so no setup change is required.
## 2. Impacted Files
- **To Modify:**
- `frontend/src/app/features/admin/categories/categories.component.ts` — include the submitted abbreviation in user-category update requests, preserve the modal on failure, and route save/upload errors to form-modal state rather than delete-modal state.
- `frontend/src/app/features/admin/categories/category-form-modal.component.ts` — accept and render a clear save/upload error and accurately expose submission state while the parent performs the async operation.
- `backend/src/modules/uploads/uploads.controller.ts` — reject undecodable/non-image category-icon buffers before any filesystem write instead of falling back to raw bytes.
- `tests/frontend/admin-categories-form-modal.spec.ts` — add minimal logic assertions for user-category abbreviation editability/submission and form error-state contract as appropriate to the extracted helpers.
- `tests/backend/uploads.spec.ts` — replace the legacy text-file success expectation with a real valid image success case and add focused corrupt/non-image rejection checks that assert no overwrite or new file occurs.
- **To Create:** None.
## 3. Proposed Changes
1. **Database / Schema Migration:** No database or migration changes. Retain the existing `UpdateCategorySchema.abbreviation` field and `AdminCategoriesService.update()` behavior, which uppercases non-system abbreviations, rejects duplicates, and protects system abbreviations. After a successful client save, the existing reload and defensive sort will move the renamed row into alphabetical order.
2. **Backend Logic & APIs:**
- In the existing `POST /api/v1/uploads/category-icon` flow, treat Sharp decode/resize failure as a `BadRequestException` with a clear image-validation message; do not use multipart MIME type, filename extension, or browser `accept` metadata as proof of validity.
- Decode and normalize the in-memory buffer completely to a 128x128 PNG before creating directories or calling `writeFileSync`. Only after Sharp succeeds should the deterministic `{categoryId}.png` path be overwritten. This preserves any previously valid icon when a replacement is corrupt and prevents partial/new files for invalid uploads.
- Return the existing successful response contract (`publicUrl`, `width: 128`, `height: 128`, `mimeType: image/png`, plus current metadata fields) so `AdminService.uploadCategoryIcon()` remains compatible. Keep the existing size and missing-file checks unchanged.
- Keep category persistence separate from upload persistence: on edit, the frontend must not dispatch the category update after upload rejection, so `updatedAt` and other fields remain unchanged. On create, retain the current create-then-upload architecture; if upload fails, no image or icon-path update is created, although the category row already created by the existing API remains. Do not introduce a database migration or new multipart category-creation endpoint outside this job's reported edit/upload defects.
3. **Frontend UI Integration:**
- Extend the edit branch of `AdminCategoriesComponent.onFormSubmit()` to pass `abbreviation: payload.abbreviation` alongside name, description, and icon path. The modal already uppercases the emitted value, while the server remains the canonical normalizer and protects system rows.
- Preserve the current `syncCategoryForm()` contract: user rows remain editable (`readonly=false`) and system rows remain locked (`readonly=true`). Do not mark user-category abbreviation readonly, because the required behavior is to persist it rather than silently discard it.
- Add dedicated category-form error state in the container, clear it when opening/closing or beginning a submission, and bind it into the form modal. On upload or update failure, keep the edit modal open, show the server's clear error, and leave the existing list/icon unchanged; do not reuse `deleteError`, which is only rendered by the delete modal.
- Coordinate a parent-managed saving flag with the modal so repeated submissions are disabled during upload/update and reset in `finally`. The modal should render the bound error using a stable test id and continue to emit a typed `CategoryFormSubmit`; no HTTP calls move into the presentational component.
## 4. Test Strategy
- **Target Unit Test File:**
- `tests/backend/uploads.spec.ts`: use Sharp to generate one tiny valid image and verify category-icon normalization succeeds; upload a plain-text payload and a corrupt PNG using a deterministic `categoryId`, expect HTTP 400 with the validation message, and assert the target icon is absent or that pre-existing valid bytes are unchanged.
- `tests/frontend/admin-categories-form-modal.spec.ts`: retain pure, CLI-only tests and add only the core contract checks needed to prove a non-system edit stays writable and produces/forwards an uppercase abbreviation, plus that a supplied save error is exposed without closing/resetting the form. If submission mapping is not currently isolatable, extract a small typed pure helper from the modal/container rather than building a large TestBed environment.
- **Mocking Strategy:** Backend endpoint tests use the real Nest route, Sharp, and filesystem boundary in the suite's isolated temporary upload directory, with generated buffers and explicit cleanup; they do not mock image decoding or require `/data`. Frontend tests mock no browser UI and exercise exported pure helpers/typed payload mapping directly. Run all tests from `/repo` with `npm test`; then run the existing workspace builds (`npm --workspace frontend run build` and `npm --workspace backend run build`) as type/build verification. No visual confirmation or external services are required.
+43
View File
@@ -0,0 +1,43 @@
# Implementation Plan: Data and run
## 1. Architectural Reconnaissance
- **Codebase style & conventions:** npm-workspace TypeScript monorepo with a NestJS 10 backend and Angular 17 standalone frontend. Runtime configuration is validated centrally with Zod in `backend/src/config/env.schema.ts` and consumed through Nest `ConfigService`; filesystem paths are normalized with Node `path.resolve`, and root scripts delegate to workspace scripts. The feature is not already implemented: persistent defaults still point at `/data/hipctf`, and the root package has no `dev` script.
- **Data Layer:** TypeORM with `better-sqlite3`. `DatabaseModule` creates the database parent directory before connecting, `DatabaseInitService` runs migrations and seeds, and uploads are written and served from the configured upload directory. No schema migration is required; this job changes only default filesystem locations and development process orchestration.
- **Test Framework & Structure:** Jest 29 with `ts-jest`, configured in `tests/jest.config.js` as backend and frontend projects. Tests live under dedicated `tests/backend` and `tests/frontend` folders and all run from the repository root with `npm test`. Add only focused configuration/script contract tests; no browser, UI, database file, port, or long-running process is required.
- **Required Tools & Dependencies:** Existing Node.js/npm workspace tooling, Angular CLI, `ts-node`, TypeScript, Jest, and `better-sqlite3` remain required. Add the root development dependency `concurrently` (and update `package-lock.json`) so one root command can supervise both long-running workspaces and propagate shutdown/failure. `setup.sh` already installs root/workspace dependencies via `npm ci`; replace its obsolete absolute `/data/hipctf/uploads` preparation with project-local `./data/uploads` preparation. No global CLI or new system package is required.
## 2. Impacted Files
- **To Modify:**
- `backend/src/config/env.schema.ts` — change validated `DATABASE_PATH` and `UPLOAD_DIR` defaults to `./data/db.sqlite` and `./data/uploads`.
- `backend/src/database/database.module.ts` — align the `ConfigService.get` database fallback with the canonical project-local database default.
- `backend/src/database/database-init.service.ts` — align `getDbPath()` fallback with the canonical project-local database default.
- `backend/src/main.ts` — align startup upload-directory fallback with the canonical project-local upload default.
- `backend/src/modules/uploads/uploads.controller.ts` — align upload controller fallback with the project-local upload default.
- `backend/src/common/utils/upload.ts` — align Multer destination fallback with the project-local upload default.
- `backend/src/frontend/uploads-static.middleware.ts` — align static-upload middleware fallback with the project-local upload default.
- `package.json` — add root `dev` orchestration for backend and frontend and retain `start:dev` as a compatible alias if desired.
- `package-lock.json` — lock the new process-runner dependency and root package metadata.
- `setup.sh` — create `./data/uploads` relative to the repository root instead of `/data/hipctf/uploads`, while retaining dependency installation and builds.
- `.gitignore` — ignore the root runtime `data/` directory rather than the unrelated `backend/data/` path, preventing local database/uploads from entering source control.
- `tests/backend/env-schema.spec.ts` — assert the exact project-local data defaults returned by the Zod schema.
- **To Create:**
- `tests/backend/root-dev-script.spec.ts` — focused static contract test for root development orchestration and both workspace targets.
## 3. Proposed Changes
1. **Database / Schema Migration:**
- Do not add a TypeORM migration or alter tables. Update the canonical default database location from `/data/hipctf/db.sqlite` to `./data/db.sqlite`.
- Replace every hard-coded database fallback outside the schema with the same value so direct/unit construction cannot silently revert to `/data` when configuration is absent.
- Preserve environment overrides such as `DATABASE_PATH=:memory:` in tests and custom deployment paths; only defaults change.
2. **Backend Logic & APIs:**
- Change the canonical default upload location from `/data/hipctf/uploads` to `./data/uploads` in the environment schema and every defensive `ConfigService.get` fallback used by startup, upload handling, Multer storage, and static serving.
- Continue resolving relative paths through `path.resolve`; when launched through the root scripts, `./data` resolves from the project root. Preserve recursive directory creation in `DatabaseModule`, `main.ts`, upload helpers, and middleware.
- Update `setup.sh` to derive/use the repository-local data directory and create `data/uploads`, avoiding writes to the containers shared `/data` volume for normal application defaults. Keep setup idempotent.
- Update `.gitignore` for `/data/` so generated SQLite and uploaded files remain local runtime artifacts.
- Add `concurrently` as a root development dependency and define `npm run dev` to launch `npm --workspace backend run start:dev` and `npm --workspace frontend run start` in parallel, with clear process names and kill-on-peer-failure/shutdown behavior. Point the existing root `start:dev` script at `npm run dev` to avoid two conflicting development entry points.
3. **Frontend UI Integration:**
- No Angular source or API URL changes are needed. The Angular dev server remains on port 4200 while the Nest API remains on port 3000; both become available from one root command.
- Because the frontend uses relative `/api` and `/uploads` URLs and no Angular dev-server proxy is currently configured, add a development proxy configuration (and reference it from Angulars `serve` target) routing `/api` and `/uploads` to `http://localhost:3000`. This makes the concurrently started frontend functional rather than merely starting two disconnected processes, including credentialed API calls and SSE streams.
## 4. Test Strategy
- **Target Unit Test File:** Extend `tests/backend/env-schema.spec.ts` with one success-path assertion that empty input resolves to `DATABASE_PATH === './data/db.sqlite'` and `UPLOAD_DIR === './data/uploads'`. Add `tests/backend/root-dev-script.spec.ts` to read root `package.json` and `frontend/angular.json`, asserting that `dev` invokes both workspace development commands and the Angular development serve target references the backend proxy. Both tests execute through the existing single root `npm test` command.
- **Mocking Strategy:** No external boundaries need heavy mocks. Parse checked-in JSON directly with Node filesystem APIs; do not spawn `npm run dev`, bind ports, launch Angular/Nest, touch `/data`, or create persistent files. The environment-schema test calls the pure Zod parser. During implementation, follow Red-Green-Refactor, then verify with `npm test`, backend/frontend builds, and the repositorys available lint/typecheck commands (no lint script currently exists; the builds provide TypeScript compilation checks).
+1 -1
View File
@@ -63,7 +63,7 @@ export function buildMulter(config: ConfigService, opts: MulterOptions = {}): mu
const limits = opts.fileSize !== undefined
? { fileSize: opts.fileSize }
: buildUploadLimits(config);
const dest = opts.destination ?? path.resolve(config.get<string>('UPLOAD_DIR', '/data/hipctf/uploads'));
const dest = opts.destination ?? path.resolve(config.get<string>('UPLOAD_DIR', './data/uploads'));
if (!fs.existsSync(dest)) fs.mkdirSync(dest, { recursive: true });
return multer({
storage: multer.diskStorage({
+2 -2
View File
@@ -4,8 +4,8 @@ export const envSchema = z.object({
NODE_ENV: z.enum(['development', 'production', 'test']).default('development'),
PORT: z.coerce.number().int().positive().default(3000),
DATABASE_PATH: z.string().min(1).default('/data/hipctf/db.sqlite'),
UPLOAD_DIR: z.string().min(1).default('/data/hipctf/uploads'),
DATABASE_PATH: z.string().min(1).default('./data/db.sqlite'),
UPLOAD_DIR: z.string().min(1).default('./data/uploads'),
THEMES_DIR: z.string().min(1).default('./themes'),
FRONTEND_DIST: z.string().min(1).default('../frontend/dist'),
@@ -113,6 +113,6 @@ export class DatabaseInitService implements OnApplicationBootstrap {
}
getDbPath(): string {
return this.config.get<string>('DATABASE_PATH', '/data/hipctf/db.sqlite');
return this.config.get<string>('DATABASE_PATH', './data/db.sqlite');
}
}
+1 -1
View File
@@ -44,7 +44,7 @@ const MIGRATIONS = [
imports: [ConfigModule],
inject: [ConfigService],
useFactory: (config: ConfigService) => {
const dbPath = config.get<string>('DATABASE_PATH', '/data/hipctf/db.sqlite');
const dbPath = config.get<string>('DATABASE_PATH', './data/db.sqlite');
const isMemory = dbPath === ':memory:' || dbPath.startsWith('file:');
if (!isMemory) {
const dir = path.dirname(dbPath);
@@ -9,7 +9,7 @@ export class UploadsStaticMiddleware implements NestMiddleware {
private rootDir = '/tmp';
constructor(config: ConfigService) {
const dir = path.resolve(config.get<string>('UPLOAD_DIR', '/data/hipctf/uploads'));
const dir = path.resolve(config.get<string>('UPLOAD_DIR', './data/uploads'));
if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true });
this.rootDir = dir;
}
+1 -1
View File
@@ -28,7 +28,7 @@ async function bootstrap(): Promise<void> {
.map((s) => s.trim())
.filter(Boolean);
const bodyLimit = config.get<string>('BODY_SIZE_LIMIT', '1mb');
const uploadDir = path.resolve(config.get<string>('UPLOAD_DIR', '/data/hipctf/uploads'));
const uploadDir = path.resolve(config.get<string>('UPLOAD_DIR', './data/uploads'));
const tlsEnabled = config.get<boolean>('TLS_ENABLED', false);
const frontendDist = path.resolve(config.get<string>('FRONTEND_DIST', '../frontend/dist'));
@@ -19,7 +19,7 @@ export class UploadsController {
private readonly globalLimit: number;
constructor(private readonly config: ConfigService) {
this.uploadDir = path.resolve(this.config.get<string>('UPLOAD_DIR', '/data/hipctf/uploads'));
this.uploadDir = path.resolve(this.config.get<string>('UPLOAD_DIR', './data/uploads'));
this.globalLimit = parseUploadSizeLimit(this.config.get<string>('UPLOAD_SIZE_LIMIT', '50mb'));
}
+3
View File
@@ -39,6 +39,9 @@
},
"serve": {
"builder": "@angular-devkit/build-angular:dev-server",
"options": {
"proxyConfig": "proxy.conf.json"
},
"configurations": {
"production": { "buildTarget": "hipctf:build:production" },
"development": { "buildTarget": "hipctf:build:development" }
+14
View File
@@ -0,0 +1,14 @@
{
"/api": {
"target": "http://localhost:3000",
"secure": false,
"changeOrigin": true,
"ws": false
},
"/uploads": {
"target": "http://localhost:3000",
"secure": false,
"changeOrigin": true,
"ws": false
}
}
+55
View File
@@ -15,6 +15,7 @@
"@types/jest": "^29.5.12",
"@types/jsdom": "^28.0.3",
"@types/node": "^20.11.0",
"concurrently": "^9.2.4",
"jest": "^29.7.0",
"jest-environment-jsdom": "^29.7.0",
"jsdom": "^29.1.1",
@@ -7899,6 +7900,60 @@
"typedarray": "^0.0.6"
}
},
"node_modules/concurrently": {
"version": "9.2.4",
"resolved": "https://registry.npmjs.org/concurrently/-/concurrently-9.2.4.tgz",
"integrity": "sha512-TZ0CEhyzvFjgtAvHTusDMgj7wNdihCh7LLLrzdUOXIhdlnL2JBBGA9eJxR24rtqgmdjh3OA3hrN1rCHj6HM8qA==",
"dev": true,
"license": "MIT",
"dependencies": {
"chalk": "4.1.2",
"rxjs": "7.8.2",
"shell-quote": "1.9.0",
"supports-color": "8.1.1",
"tree-kill": "1.2.2",
"yargs": "17.7.2"
},
"bin": {
"conc": "dist/bin/concurrently.js",
"concurrently": "dist/bin/concurrently.js"
},
"engines": {
"node": ">=18"
},
"funding": {
"url": "https://github.com/open-cli-tools/concurrently?sponsor=1"
}
},
"node_modules/concurrently/node_modules/shell-quote": {
"version": "1.9.0",
"resolved": "https://registry.npmjs.org/shell-quote/-/shell-quote-1.9.0.tgz",
"integrity": "sha512-Iov+JwFv/2HcTpcwNMKd8+IWNb8tboQJNQTkAY/LLVK7gGH9jy+LGkVqPxfekHl+yMmiqXszdGWXgkfml7hjqA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/concurrently/node_modules/supports-color": {
"version": "8.1.1",
"resolved": "https://registry.npmjs.org/supports-color/-/supports-color-8.1.1.tgz",
"integrity": "sha512-MpUEN2OodtUzxvKQl72cUF7RQ5EiHsGvSsVG0ia9c5RbWGL2CI4C7EpPS8UTBIplnlzZiNuV56w+FuNxy3ty2Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"has-flag": "^4.0.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/chalk/supports-color?sponsor=1"
}
},
"node_modules/connect-history-api-fallback": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/connect-history-api-fallback/-/connect-history-api-fallback-2.0.0.tgz",
+4
View File
@@ -11,6 +11,9 @@
"build": "npm --workspace frontend run build && npm --workspace backend run build",
"start": "node backend/dist/main.js",
"start:dev": "npm --workspace backend run start:dev",
"dev": "concurrently --kill-others-on-fail --names \"backend,frontend\" --prefix-colors \"auto\" \"npm --workspace backend run start:dev\" \"npm --workspace frontend run start\"",
"dev:backend": "npm --workspace backend run start:dev",
"dev:frontend": "npm --workspace frontend run start",
"test": "jest --config tests/jest.config.js",
"test:backend": "jest --config tests/jest.config.js --selectProjects backend",
"test:frontend": "jest --config tests/jest.config.js --selectProjects frontend",
@@ -21,6 +24,7 @@
"@types/jest": "^29.5.12",
"@types/jsdom": "^28.0.3",
"@types/node": "^20.11.0",
"concurrently": "^9.1.0",
"jest": "^29.7.0",
"jest-environment-jsdom": "^29.7.0",
"jsdom": "^29.1.1",
+3 -3
View File
@@ -1,8 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail
echo "[setup] Ensuring /data/hipctf exists..."
mkdir -p /data/hipctf/uploads
echo "[setup] Ensuring ./data/uploads exists..."
mkdir -p ./data/uploads
echo "[setup] Installing root dependencies..."
npm ci --no-audit --no-fund || npm install --no-audit --no-fund
@@ -16,4 +16,4 @@ npm --workspace frontend run build
echo "[setup] Building NestJS backend..."
npm --workspace backend run build
echo "[setup] Done. Start with: npm start"
echo "[setup] Done. Start with: npm start (dev mode: npm run dev)"
+12
View File
@@ -6,6 +6,18 @@ describe('envSchema', () => {
expect(r.success).toBe(true);
});
it('defaults DATABASE_PATH to ./data/db.sqlite', () => {
const r = envSchema.safeParse({});
expect(r.success).toBe(true);
expect((r.data as any).DATABASE_PATH).toBe('./data/db.sqlite');
});
it('defaults UPLOAD_DIR to ./data/uploads', () => {
const r = envSchema.safeParse({});
expect(r.success).toBe(true);
expect((r.data as any).UPLOAD_DIR).toBe('./data/uploads');
});
it('rejects invalid PORT', () => {
const r = envSchema.safeParse({ PORT: 'not-a-number' });
expect(r.success).toBe(false);
+26
View File
@@ -0,0 +1,26 @@
import { readFileSync } from 'node:fs';
import * as path from 'node:path';
describe('Root dev script orchestration', () => {
it('exposes a `dev` script that runs both backend and frontend workspaces', () => {
const pkg = JSON.parse(
readFileSync(path.join(__dirname, '..', '..', 'package.json'), 'utf8'),
);
expect(typeof pkg.scripts?.dev).toBe('string');
expect(pkg.scripts.dev).toMatch(/workspace\s+backend/);
expect(pkg.scripts.dev).toMatch(/workspace\s+frontend/);
});
it('configures the Angular dev server to proxy /api and /uploads to the Nest backend', () => {
const angularJson = JSON.parse(
readFileSync(
path.join(__dirname, '..', '..', 'frontend', 'angular.json'),
'utf8',
),
);
const serve = angularJson.projects?.hipctf?.architect?.serve;
expect(serve).toBeDefined();
expect(typeof serve.options?.proxyConfig).toBe('string');
expect(serve.options.proxyConfig).toMatch(/proxy/);
});
});