134 lines
9.6 KiB
Markdown
134 lines
9.6 KiB
Markdown
---
|
|
type: architecture
|
|
title: System Architecture
|
|
description: Component map of 10Backward — how configuration, blog synchronization, anniversary matching, logging, the pipeline runner, state persistence, and container entrypoint are wired together.
|
|
tags: [architecture, overview]
|
|
timestamp: 2026-08-04T18:55:00Z
|
|
---
|
|
|
|
# Overview
|
|
|
|
`10Backward` is a Mastodon daily-throwback bot that runs as a single
|
|
containerised cron job. Job 1086 adds anniversary matching across Jekyll
|
|
posts, backed by the blog clone/pull workflow from Job 1084 and the
|
|
structured logging, retry-capable runner, and deduplication state from
|
|
Job 1083. The current pipeline identifies matching posts, publishes a
|
|
single combined Mastodon status for the new matches, and records the
|
|
published identifiers in `posted.json`.
|
|
|
|
The container boots, validates environment configuration, renders a
|
|
`/etc/cron.d/tenbackward` entry that fires once per day at the
|
|
configured `RUN_AT`, and then runs `python -m tenbackward` in the
|
|
foreground of the `cron` process. Each scheduled invocation calls
|
|
`tenbackward.main.main()`, which:
|
|
|
|
1. Installs the JSON logging formatter on the root logger.
|
|
2. Loads and validates the `Config`.
|
|
3. Ensures the data directory exists.
|
|
4. Emits a `startup` log line.
|
|
5. Executes one pipeline pass wrapped in a retry loop:
|
|
`ensure_repo` → `find_anniversary_matches` → `PostedStore` dedupe →
|
|
`publish_mastodon` → `PostedStore.mark_posted_many`.
|
|
6. Emits a `run_complete` summary (skipped silently if there were no candidates).
|
|
|
|
# Components
|
|
|
|
| Component | Responsibility |
|
|
|--------------------------|------------------------------------------------------------------------------------------------------------|
|
|
| `entrypoint.sh` | Validates required env vars, renders the cron file from `RUN_AT`/`TZ`, then `exec cron -f`. |
|
|
| `tenbackward.main` | CLI entry point; orchestrates startup logging, retry-wrapped pipeline pass, and run summary. |
|
|
| `tenbackward.config` | Loads `.env` + process env, applies defaults, validates schema, produces a typed `Config` dataclass. |
|
|
| `tenbackward.logging_setup` | JSON formatter (one log record per line), secret redaction, and `log_startup` / `log_error` / `log_run_summary` helpers. |
|
|
| `tenbackward.state` | `PostedStore` — self-contained dedup store. `posted.json` holds `{"posted": [str, ...]}`; mutations are serialised by an `fcntl.flock` on a sibling lock file, writes go through a temp-file replace, and `load()` auto-creates an empty list when the file is missing. |
|
|
| `tenbackward.blog` | GitPython `ensure_repo()` — clones the configured blog repo on first run, fast-forwards it via `pull --ff-only` thereafter, with exponential-backoff retries on transient network errors. |
|
|
| `tenbackward.matching` | Walks `_posts/blog/**/*.md`, parses Jekyll front matter and filenames, and yields posts exactly ten years before the current date using Berlin-time and leap-day rules. |
|
|
| `tenbackward.publishing` | `publish_mastodon()` — the single success boundary between the runner and the Mastodon HTTP API. Composes a combined status for the day's matches, validates the 500-character limit, and posts via `mastodon.Mastodon.status_post`. Raises `PublishError` on composition, length, or API failure; the runner treats it like any other pipeline exception. |
|
|
| `/etc/cron.d/tenbackward`| Rendered cron file. One daily line that `cd /app` and runs `python -m tenbackward`. |
|
|
|
|
# Communication & Wiring
|
|
|
|
```
|
|
+-------------------+ env vars +-------------------------+
|
|
| entrypoint | -------------------> | config.load_config |
|
|
| (.sh) | | (Config dataclass) |
|
|
+---------+---------+ +-----------+-------------+
|
|
| |
|
|
| installs cron file | feeds
|
|
v v
|
|
+-------------------+ +-------------------------+
|
|
| /etc/cron.d/ | -- daily fires --> | tenbackward.main.main |
|
|
| tenbackward | | - configure logging |
|
|
+-------------------+ | - retry-wrapped pass |
|
|
+-----------+-------------+
|
|
|
|
|
v
|
|
+-------------------------+
|
|
| _run_with_retry(...) |
|
|
| -> _run_once(...) |
|
|
| -> ensure_repo(...) |
|
|
| -> matching |
|
|
| find_anniversary_|
|
|
| matches |
|
|
| -> PostedStore |
|
|
| {is_posted, |
|
|
| mark_posted_many}|
|
|
| -> publishing |
|
|
| publish_mastodon |
|
|
| -> PostedStore |
|
|
| mark_posted_many |
|
|
+-------------------------+
|
|
|
|
|
v
|
|
+-------------------------+
|
|
| Mastodon HTTP API |
|
|
| status_post(...) |
|
|
+-------------------------+
|
|
```
|
|
|
|
* **Env → Config.** `load_config()` merges a `.env` file (when
|
|
present) under the process environment via `dotenv_values` + `load_dotenv`,
|
|
applies defaults, then validates the merged map before producing the
|
|
`Config` dataclass.
|
|
* **Cron → main.** Each scheduled tick re-runs `python -m tenbackward`,
|
|
so every run is a fresh interpreter invocation.
|
|
* **Matching → publishing.** `find_anniversary_matches` yields
|
|
`MatchedPost` objects; the runner dedupes against `PostedStore`, then
|
|
passes the unposted objects to `publish_mastodon`. See
|
|
[Mastodon Publishing](/architecture/mastodon-publishing.md).
|
|
* **Pipeline state.** `_run_once` reads `posted.json` via
|
|
`PostedStore.is_posted()` for dedupe, then writes it back via
|
|
`PostedStore.mark_posted_many()` **only after** `publish_mastodon`
|
|
returns. The store serialises concurrent runs with an
|
|
`fcntl.flock` on a sibling lock file and writes via temp-file
|
|
rename.
|
|
|
|
# Key Files
|
|
|
|
| Path | Responsibility |
|
|
|-------------------------------------|---------------------------------------------------------------------------------|
|
|
| `/repo/entrypoint.sh` | Bootstraps cron; validates required env vars; renders `/etc/cron.d/tenbackward`. |
|
|
| `/repo/src/tenbackward/main.py` | CLI entry, retry wrapper, pipeline counters. |
|
|
| `/repo/src/tenbackward/config.py` | Env merging, defaults, validation, `Config` dataclass, `ConfigError`. |
|
|
| `/repo/src/tenbackward/logging_setup.py` | JSON formatter, secret redaction, structured event helpers. |
|
|
| `/repo/src/tenbackward/state.py` | `PostedStore` + module helpers (`load_posted`, `is_posted`, `mark_posted`, `mark_posted_many`); list-shaped JSON, `fcntl.flock`, atomic temp-file replace, env-var data dir. |
|
|
| `/repo/src/tenbackward/blog.py` | `ensure_repo()` — GitPython clone + fast-forward pull with `2^n` retry/backoff; raises `BlogRepoError` on local modifications or exhausted retries. |
|
|
| `/repo/src/tenbackward/matching.py` | Jekyll post discovery, front matter parsing, anniversary and leap-day matching, and canonical URL construction. |
|
|
| `/repo/src/tenbackward/publishing.py` | `publish_mastodon()` — composes a combined Mastodon status, validates the 500-character limit, posts via `mastodon.Mastodon.status_post`. Pure helpers `build_status_text` and `validate_status` keep network code out of tests. |
|
|
| `/repo/src/tenbackward/__main__.py` | Module entrypoint that invokes `tenbackward.main.main()`. |
|
|
| `/repo/.env.example` | Canonical list of environment variables. |
|
|
| `/repo/docker-compose.yml` | Service definition; binds env vars from the host `.env`. |
|
|
| `/repo/Dockerfile` | Builds the runtime image. |
|
|
| `/repo/crontab/tenbackward.cron` | Cron template shipped in the image. |
|
|
| `/repo/crontab/install-cron.sh` | Installs the rendered cron file with mode 0644; refuses placeholder leftovers. |
|
|
| `/repo/tests/` | Pytest suite covering config, logging, runner, cron, and entrypoint validation. |
|
|
|
|
# Related
|
|
|
|
* [Config Schema](/architecture/config-schema.md)
|
|
* [Logging & Run Summary](/architecture/logging.md)
|
|
* [Pipeline Runner](/architecture/pipeline-runner.md)
|
|
* [Anniversary Matching](/architecture/anniversary-matching.md)
|
|
* [Mastodon Publishing](/architecture/mastodon-publishing.md)
|
|
* [Environment Variable Setup](/operations/environment-setup.md)
|
|
* [Cron Lifecycle](/operations/cron-lifecycle.md)
|