10Backward/docs/architecture/system-overview.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
architecture	System Architecture	Component map of 10Backward — how configuration, blog synchronization, anniversary matching, logging, the pipeline runner, state persistence, and container entrypoint are wired together.	architecture overview	2026-08-04T17:51:00Z

Overview

10Backward is a Mastodon daily-throwback bot that runs as a single containerised cron job. Job 1086 adds anniversary matching across Jekyll posts, backed by the blog clone/pull workflow from Job 1084 and the structured logging, retry-capable runner, and deduplication state from Job 1083. The current pipeline identifies matching post paths and records those identifiers; Mastodon publishing is still a future integration.

The container boots, validates environment configuration, renders a /etc/cron.d/tenbackward entry that fires once per day at the configured RUN_AT, and then runs python -m tenbackward in the foreground of the cron process. Each scheduled invocation calls tenbackward.main.main(), which:

1. Installs the JSON logging formatter on the root logger.
2. Loads and validates the Config.
3. Ensures the data directory exists.
4. Emits a startup log line.
5. Executes one pipeline pass wrapped in a retry loop.
6. Emits a run_complete summary (skipped silently if there were no candidates).

Components

Component	Responsibility
entrypoint.sh	Validates required env vars, renders the cron file from RUN_AT/TZ, then exec cron -f.
tenbackward.main	CLI entry point; orchestrates startup logging, retry-wrapped pipeline pass, and run summary.
tenbackward.config	Loads .env + process env, applies defaults, validates schema, produces a typed Config dataclass.
tenbackward.logging_setup	JSON formatter (one log record per line), secret redaction, and log_startup / log_error / log_run_summary helpers.
tenbackward.state	PostedStore — self-contained dedup store. posted.json holds {"posted": [str, ...]}; mutations are serialised by an fcntl.flock on a sibling lock file, writes go through a temp-file replace, and load() auto-creates an empty list when the file is missing.
tenbackward.blog	GitPython ensure_repo() — clones the configured blog repo on first run, fast-forwards it via pull --ff-only thereafter, with exponential-backoff retries on transient network errors.
tenbackward.matching	Walks _posts/blog/**/*.md, parses Jekyll front matter and filenames, and yields posts exactly ten years before the current date using Berlin-time and leap-day rules.
/etc/cron.d/tenbackward	Rendered cron file. One daily line that cd /app and runs python -m tenbackward.

Communication & Wiring

+-------------------+      env vars       +-------------------------+
|     entrypoint    | -------------------> |   config.load_config    |
|     (.sh)         |                     |   (Config dataclass)    |
+---------+---------+                     +-----------+-------------+
          |                                           |
          |  installs cron file                        |  feeds
          v                                           v
+-------------------+                     +-------------------------+
| /etc/cron.d/       | -- daily fires --> |   tenbackward.main.main |
| tenbackward        |                    |   - configure logging   |
+-------------------+                    |   - retry-wrapped pass  |
                                         +-----------+-------------+
                                                     |
                                                     v
+-------------------------+
                                          | _run_with_retry(...)    |
                                          |  -> _run_once(...)      |
                                          |     -> PostedStore.     |
                                          |        {is_posted,      |
                                          |         mark_posted_many}|
                                          +-------------------------+

• Env → Config. load_config() merges a .env file (when present) under the process environment via dotenv_values + load_dotenv, applies defaults, then validates the merged map before producing the Config dataclass.
• Cron → main. Each scheduled tick re-runs python -m tenbackward, so every run is a fresh interpreter invocation.
• Pipeline state. _run_once reads posted.json via PostedStore.is_posted() for dedupe, then writes it back via PostedStore.mark_posted_many() only when at least one new post was recorded. The store serialises concurrent runs with an fcntl.flock on a sibling lock file and writes via temp-file rename.

Key Files

Path	Responsibility
/repo/entrypoint.sh	Bootstraps cron; validates required env vars; renders /etc/cron.d/tenbackward.
/repo/src/tenbackward/main.py	CLI entry, retry wrapper, pipeline counters.
/repo/src/tenbackward/config.py	Env merging, defaults, validation, Config dataclass, ConfigError.
/repo/src/tenbackward/logging_setup.py	JSON formatter, secret redaction, structured event helpers.
/repo/src/tenbackward/state.py	PostedStore + module helpers (load_posted, is_posted, mark_posted, mark_posted_many); list-shaped JSON, fcntl.flock, atomic temp-file replace, env-var data dir.
/repo/src/tenbackward/blog.py	ensure_repo() — GitPython clone + fast-forward pull with 2^n retry/backoff; raises BlogRepoError on local modifications or exhausted retries.
/repo/src/tenbackward/matching.py	Jekyll post discovery, front matter parsing, anniversary and leap-day matching, and canonical URL construction.
/repo/src/tenbackward/__main__.py	Module entrypoint that invokes tenbackward.main.main().
/repo/.env.example	Canonical list of environment variables.
/repo/docker-compose.yml	Service definition; binds env vars from the host .env.
/repo/Dockerfile	Builds the runtime image.
/repo/crontab/tenbackward.cron	Cron template shipped in the image.
/repo/crontab/install-cron.sh	Installs the rendered cron file with mode 0644; refuses placeholder leftovers.
/repo/tests/	Pytest suite covering config, logging, runner, cron, and entrypoint validation.

Related

• Config Schema
• Logging & Run Summary
• Pipeline Runner
• Anniversary Matching
• Environment Variable Setup
• Cron Lifecycle