m0rph3us1987/10Backward
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

AI Implementation feature(1086): Posted-Anniversary Dedup Store (#5)

Browse Source
This commit was merged in pull request #5.
This commit is contained in:
m0rph3us1987
2026-08-04 18:35:28 +00:00
parent 72c2369160
commit 7d39f3a339
10 changed files with 516 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/system-overview.md
+24 -13
View File
@@ -1,7 +1,7 @@
---
type: architecture
title: System Architecture
description: Component map of 10Backward — how config, logging, the pipeline runner, state persistence, and container entrypoint are wired together.
description: Component map of 10Backward — how configuration, blog synchronization, anniversary matching, logging, the pipeline runner, state persistence, and container entrypoint are wired together.
tags: [architecture, overview]
timestamp: 2026-08-04T17:51:00Z
---
@@ -9,8 +9,11 @@ timestamp: 2026-08-04T17:51:00Z
# Overview
`10Backward` is a Mastodon daily-throwback bot that runs as a single
containerised cron job. Job 1083 adds a structured logging layer and a
retry-capable pipeline runner on top of the scaffold shipped by Job 1082.
containerised cron job. Job 1086 adds anniversary matching across Jekyll
posts, backed by the blog clone/pull workflow from Job 1084 and the
structured logging, retry-capable runner, and deduplication state from
Job 1083. The current pipeline identifies matching post paths and records
those identifiers; Mastodon publishing is still a future integration.
The container boots, validates environment configuration, renders a
`/etc/cron.d/tenbackward` entry that fires once per day at the
@@ -33,8 +36,9 @@ foreground of the `cron` process. Each scheduled invocation calls
| `tenbackward.main` | CLI entry point; orchestrates startup logging, retry-wrapped pipeline pass, and run summary. |
| `tenbackward.config` | Loads `.env` + process env, applies defaults, validates schema, produces a typed `Config` dataclass. |
| `tenbackward.logging_setup` | JSON formatter (one log record per line), secret redaction, and `log_startup` / `log_error` / `log_run_summary` helpers. |
| `tenbackward.state` | Read/write of `posted.json` (atomic temp-file replace). |
| `tenbackward.state` | `PostedStore` — self-contained dedup store. `posted.json` holds `{"posted": [str, ...]}`; mutations are serialised by an `fcntl.flock` on a sibling lock file, writes go through a temp-file replace, and `load()` auto-creates an empty list when the file is missing. |
| `tenbackward.blog` | GitPython `ensure_repo()` — clones the configured blog repo on first run, fast-forwards it via `pull --ff-only` thereafter, with exponential-backoff retries on transient network errors. |
| `tenbackward.matching` | Walks `_posts/blog/**/*.md`, parses Jekyll front matter and filenames, and yields posts exactly ten years before the current date using Berlin-time and leap-day rules. |
| `/etc/cron.d/tenbackward`| Rendered cron file. One daily line that `cd /app` and runs `python -m tenbackward`. |
# Communication & Wiring
@@ -54,11 +58,13 @@ foreground of the `cron` process. Each scheduled invocation calls
+-----------+-------------+
|
v
+-------------------------+
| _run_with_retry(...) |
| -> _run_once(...) |
| -> state.{load,save}|
+-------------------------+
+-------------------------+
| _run_with_retry(...) |
| -> _run_once(...) |
| -> PostedStore. |
| {is_posted, |
| mark_posted_many}|
+-------------------------+
```
* **Env → Config.** `load_config()` merges a `.env` file (when
@@ -68,8 +74,11 @@ foreground of the `cron` process. Each scheduled invocation calls
* **Cron → main.** Each scheduled tick re-runs `python -m tenbackward`,
so every run is a fresh interpreter invocation.
* **Pipeline state.** `_run_once` reads `posted.json` via
`state.load_posted()` for dedupe, then writes it back via
`state.save_posted()` only when at least one new post was recorded.
`PostedStore.is_posted()` for dedupe, then writes it back via
`PostedStore.mark_posted_many()` only when at least one new post
was recorded. The store serialises concurrent runs with an
`fcntl.flock` on a sibling lock file and writes via temp-file
rename.
# Key Files
@@ -79,8 +88,10 @@ foreground of the `cron` process. Each scheduled invocation calls
| `/repo/src/tenbackward/main.py` | CLI entry, retry wrapper, pipeline counters. |
| `/repo/src/tenbackward/config.py` | Env merging, defaults, validation, `Config` dataclass, `ConfigError`. |
| `/repo/src/tenbackward/logging_setup.py` | JSON formatter, secret redaction, structured event helpers. |
| `/repo/src/tenbackward/state.py` | `posted.json` read/write with atomic temp-file replace. |
| `/repo/src/tenbackward/state.py` | `PostedStore` + module helpers (`load_posted`, `is_posted`, `mark_posted`, `mark_posted_many`); list-shaped JSON, `fcntl.flock`, atomic temp-file replace, env-var data dir. |
| `/repo/src/tenbackward/blog.py` | `ensure_repo()` — GitPython clone + fast-forward pull with `2^n` retry/backoff; raises `BlogRepoError` on local modifications or exhausted retries. |
| `/repo/src/tenbackward/matching.py` | Jekyll post discovery, front matter parsing, anniversary and leap-day matching, and canonical URL construction. |
| `/repo/src/tenbackward/__main__.py` | Module entrypoint that invokes `tenbackward.main.main()`. |
| `/repo/.env.example` | Canonical list of environment variables. |
| `/repo/docker-compose.yml` | Service definition; binds env vars from the host `.env`. |
| `/repo/Dockerfile` | Builds the runtime image. |
@@ -93,6 +104,6 @@ foreground of the `cron` process. Each scheduled invocation calls
* [Config Schema](/architecture/config-schema.md)
* [Logging & Run Summary](/architecture/logging.md)
* [Pipeline Runner](/architecture/pipeline-runner.md)
* [Daily Run Guide](/guides/daily-run.md)
* [Anniversary Matching](/architecture/anniversary-matching.md)
* [Environment Variable Setup](/operations/environment-setup.md)
* [Cron Lifecycle](/operations/cron-lifecycle.md)