10Backward/docs/architecture/system-overview.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
architecture	System Architecture	Component map of 10Backward — how configuration, blog synchronization, anniversary matching, logging, the pipeline runner, state persistence, and container entrypoint are wired together.	architecture overview	2026-08-04T18:55:00Z

Overview

10Backward is a Mastodon daily-throwback bot that runs as a single containerised cron job. Job 1086 adds anniversary matching across Jekyll posts, backed by the blog clone/pull workflow from Job 1084 and the structured logging, retry-capable runner, and deduplication state from Job 1083. The current pipeline identifies matching posts, publishes a single combined Mastodon status for the new matches, and records the published identifiers in posted.json.

The container boots, validates environment configuration, renders a /etc/cron.d/tenbackward entry that fires once per day at the configured RUN_AT, and then runs python -m tenbackward in the foreground of the cron process. Each scheduled invocation calls tenbackward.main.main(), which:

1. Installs the JSON logging formatter on the root logger.
2. Loads and validates the Config.
3. Ensures the data directory exists.
4. Emits a startup log line.
5. Executes one pipeline pass wrapped in a retry loop: ensure_repo → find_anniversary_matches → PostedStore dedupe → publish_mastodon → PostedStore.mark_posted_many.
6. Emits a run_complete summary (skipped silently if there were no candidates).

Components

Component	Responsibility
entrypoint.sh	Validates required env vars, renders the cron file from RUN_AT/TZ, then exec cron -f.
tenbackward.main	CLI entry point; orchestrates startup logging, retry-wrapped pipeline pass, and run summary.
tenbackward.config	Loads .env + process env, applies defaults, validates schema, produces a typed Config dataclass.
tenbackward.logging_setup	JSON formatter (one log record per line), secret redaction, and log_startup / log_error / log_run_summary helpers.
tenbackward.state	PostedStore — self-contained dedup store. posted.json holds {"posted": [str, ...]}; mutations are serialised by an fcntl.flock on a sibling lock file, writes go through a temp-file replace, and load() auto-creates an empty list when the file is missing.
tenbackward.blog	GitPython ensure_repo() — clones the configured blog repo on first run, fast-forwards it via pull --ff-only thereafter, with exponential-backoff retries on transient network errors.
tenbackward.matching	Walks _posts/blog/**/*.md, parses Jekyll front matter and filenames, and yields posts exactly ten years before the current date using Berlin-time and leap-day rules.
tenbackward.publishing	publish_mastodon() — the single success boundary between the runner and the Mastodon HTTP API. Composes a combined status for the day's matches, validates the 500-character limit, and posts via mastodon.Mastodon.status_post. Raises PublishError on composition, length, or API failure; the runner treats it like any other pipeline exception.
/etc/cron.d/tenbackward	Rendered cron file. One daily line that cd /app and runs python -m tenbackward.

Communication & Wiring

+-------------------+      env vars       +-------------------------+
|     entrypoint    | -------------------> |   config.load_config    |
|     (.sh)         |                     |   (Config dataclass)    |
+---------+---------+                     +-----------+-------------+
          |                                           |
          |  installs cron file                        |  feeds
          v                                           v
+-------------------+                     +-------------------------+
| /etc/cron.d/       | -- daily fires --> |   tenbackward.main.main |
| tenbackward        |                    |   - configure logging   |
+-------------------+                    |   - retry-wrapped pass  |
                                          +-----------+-------------+
                                                      |
                                                      v
                                           +-------------------------+
                                           | _run_with_retry(...)    |
                                           |  -> _run_once(...)      |
                                           |     -> ensure_repo(...) |
                                           |     -> matching         |
                                           |        find_anniversary_|
                                           |        matches          |
                                           |     -> PostedStore      |
                                           |        {is_posted,      |
                                           |         mark_posted_many}|
                                           |     -> publishing       |
                                           |        publish_mastodon |
                                           |     -> PostedStore      |
                                           |        mark_posted_many |
                                           +-------------------------+
                                                      |
                                                      v
                                           +-------------------------+
                                           |   Mastodon HTTP API     |
                                           |   status_post(...)      |
                                           +-------------------------+

• Env → Config. load_config() merges a .env file (when present) under the process environment via dotenv_values + load_dotenv, applies defaults, then validates the merged map before producing the Config dataclass.
• Cron → main. Each scheduled tick re-runs python -m tenbackward, so every run is a fresh interpreter invocation.
• Matching → publishing. find_anniversary_matches yields MatchedPost objects; the runner dedupes against PostedStore, then passes the unposted objects to publish_mastodon. See Mastodon Publishing.
• Pipeline state. _run_once reads posted.json via PostedStore.is_posted() for dedupe, then writes it back via PostedStore.mark_posted_many() only after publish_mastodon returns. The store serialises concurrent runs with an fcntl.flock on a sibling lock file and writes via temp-file rename.

Key Files

Path	Responsibility
/repo/entrypoint.sh	Bootstraps cron; validates required env vars; renders /etc/cron.d/tenbackward.
/repo/src/tenbackward/main.py	CLI entry, retry wrapper, pipeline counters.
/repo/src/tenbackward/config.py	Env merging, defaults, validation, Config dataclass, ConfigError.
/repo/src/tenbackward/logging_setup.py	JSON formatter, secret redaction, structured event helpers.
/repo/src/tenbackward/state.py	PostedStore + module helpers (load_posted, is_posted, mark_posted, mark_posted_many); list-shaped JSON, fcntl.flock, atomic temp-file replace, env-var data dir.
/repo/src/tenbackward/blog.py	ensure_repo() — GitPython clone + fast-forward pull with 2^n retry/backoff; raises BlogRepoError on local modifications or exhausted retries.
/repo/src/tenbackward/matching.py	Jekyll post discovery, front matter parsing, anniversary and leap-day matching, and canonical URL construction.
/repo/src/tenbackward/publishing.py	publish_mastodon() — composes a combined Mastodon status, validates the 500-character limit, posts via mastodon.Mastodon.status_post. Pure helpers build_status_text and validate_status keep network code out of tests.
/repo/src/tenbackward/__main__.py	Module entrypoint that invokes tenbackward.main.main().
/repo/.env.example	Canonical list of environment variables.
/repo/docker-compose.yml	Service definition; binds env vars from the host .env.
/repo/Dockerfile	Builds the runtime image.
/repo/crontab/tenbackward.cron	Cron template shipped in the image.
/repo/crontab/install-cron.sh	Installs the rendered cron file with mode 0644; refuses placeholder leftovers.
/repo/tests/	Pytest suite covering config, logging, runner, cron, and entrypoint validation.

Related

• Config Schema
• Logging & Run Summary
• Pipeline Runner
• Anniversary Matching
• Mastodon Publishing
• Environment Variable Setup
• Cron Lifecycle