m0rph3us1987/10Backward
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update documentation to OKF v0.1 format

Browse Source
This commit is contained in:
OpenVelo Agent
2026-08-04 18:57:40 +00:00
parent 7d39f3a339
commit d435aa6f8f
8 changed files with 255 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/config-schema.md
+1 -1
View File
@@ -44,7 +44,7 @@ fills into the merged map **before** validation runs:
| Default key | Default value |
|-----------------------|-------------------------------------------|
| `VISIBILITY` | `public` |
| `THROWNBACK_PREFIX` | `Throwback:` |
| `THROWNBACK_PREFIX` | `Heute vor 10 Jahren:` |
| `MAX_RETRIES` | `3` |
| `TZ` | `Europe/Berlin` |
| `RUN_AT` | `09:00` |
docs/architecture/logging.md
+2 -1
View File
@@ -95,7 +95,7 @@ A pass with no candidates emits **no line at all**, matching the
## Successful run with one new post
```json
{"ts":"2026-08-04T09:00:00+00:00","level":"INFO","logger":"tenbackward","message":"startup","event":"startup","version":"0.1.0","site_url":"https://blog.example.com","run_at":"09:00","tz":"Europe/Berlin","hashtags":"#throwback,#10backward","throwback_prefix":"Throwback:"}
{"ts":"2026-08-04T09:00:00+00:00","level":"INFO","logger":"tenbackward","message":"startup","event":"startup","version":"0.1.0","site_url":"https://blog.example.com","run_at":"09:00","tz":"Europe/Berlin","hashtags":"#throwback,#10backward","throwback_prefix":"Heute vor 10 Jahren:"}
{"ts":"2026-08-04T09:00:01+00:00","level":"INFO","logger":"tenbackward","message":"run complete","event":"run_complete","scanned":1,"matched":1,"posted":1,"skipped":0,"posted_ids":["2025-08-04-post-slug"]}
```
@@ -123,3 +123,4 @@ formatter replaces it:
* [System Architecture](/architecture/system-overview.md)
* [Pipeline Runner](/architecture/pipeline-runner.md)
* [Config Schema](/architecture/config-schema.md)
* [Mastodon Publishing](/architecture/mastodon-publishing.md)
docs/architecture/mastodon-publishing.md
+174
View File
@@ -0,0 +1,174 @@
---
type: architecture
title: Mastodon Publishing
description: The single boundary between the pipeline runner and the Mastodon HTTP API — status composition, length validation, and the publish call.
tags: [publishing, mastodon, api, boundary]
timestamp: 2026-08-04T18:55:00Z
---
# Purpose
`tenbackward.publishing` is the **single success boundary** between the
pipeline orchestrator and the Mastodon HTTP API. The runner calls
`publish_mastodon()` after dedupe and before state is persisted; if
the call raises, the retry loop re-attempts and `posted.json` is left
untouched. Composition and validation are pure functions that do not
touch the network so they are unit-tested without a server.
# Public API
| Symbol | Responsibility |
|---|---|
| `MASTODON_STATUS_LIMIT` | `500` — Mastodon's hard maximum status length. |
| `PublishError` | Raised on composition failure, length overflow, or any wrapped API exception. Subclass of `RuntimeError`. |
| `build_status_text(prefix, posts, hashtags)` | Pure composer. Returns the final status string. Raises `PublishError("empty_posts: ...")` when `posts` is empty. |
| `validate_status(status, limit=MASTODON_STATUS_LIMIT)` | Pure validator. Raises `PublishError("status_too_long: len=N limit=L")` when `len(status) > limit`. Never truncates. |
| `publish_mastodon(config, posts, *, client_factory=None)` | Composes, validates, and posts. Returns the composed status string. Wraps every third-party exception in `PublishError(f"publish_failed: {ExcType}: {exc}", ) from exc`. |
| `slugify_title(title)` | Re-exported from [matching](/architecture/anniversary-matching.md); used by callers that need to derive the same URL slug the matcher emits. |
# Status Layout
`build_status_text` produces exactly one Mastodon status for any number
of matching posts, in this shape:
```text
{prefix}
{title1}
{url1}
{title2}
{url2}
...
{hashtags}
```
* The prefix comes from `Config.throwback_prefix` (default
`Heute vor 10 Jahren:`, overridable via `THROWNBACK_PREFIX`).
* Posts are sorted by `(date, path)` so output is deterministic
regardless of upstream order.
* `hashtags` is a comma-separated string; commas are collapsed to a
single space so each tag starts with `#` and no trailing comma
is emitted (empty tags are dropped).
* A trailing newline is always appended.
# Composition Examples
## Single post
```text
Heute vor 10 Jahren:
Mein erster Post
https://blog.example.com/2016/08/04/mein-erster-post/
#throwback #10backward
```
## Multiple posts on the same day
When more than one Jekyll post matches the current day, they are
**merged into one status** — the runner does not issue separate API
calls per match. Ordering is `(date, path)`, so identical dates sort
by relative path:
```text
Heute vor 10 Jahren:
Post A
https://blog.example.com/2016/08/04/a/
Post B
https://blog.example.com/2016/08/04/b/
#throwback #10backward
```
## No posts
Calling `build_status_text("...", [], "#x")` raises
`PublishError("empty_posts: cannot compose status without posts")`
before any API call. The runner only invokes the publisher when its
candidate list is non-empty, so this guard is a backstop for direct
callers and unit tests.
# Length Validation
Mastodon's API rejects statuses longer than 500 characters. The
spec requires **failing safely** — the bot must never silently
truncate content. `validate_status()` enforces this contract: the
full status is measured, and overflow raises
`PublishError("status_too_long: len=N limit=500")` *before* the API
call. The runner's retry loop treats this exactly like any other
publish failure.
# Mastodon API Call
`_post_status_via_mastodon_py()` constructs a `mastodon.Mastodon`
client with `(access_token=config.mastodon_access_token,
api_base_url=config.mastodon_base_url)` and calls
`status_post(status, visibility=config.visibility)`.
The `client_factory` keyword argument on `publish_mastodon` lets
tests inject a fake client without monkey-patching. Production
callers leave it as `None`.
# Failure Modes
| Source | Exception surfaced to runner | Cause attached? |
|---|---|---|
| Empty `posts` argument | `PublishError("empty_posts: ...")` | No (no inner exc). |
| Composed status > 500 chars | `PublishError("status_too_long: ...")` | No. |
| Any other exception from inside the boundary | `PublishError("publish_failed: {ExcType}: {exc}")` | Yes, via `raise ... from exc`. |
The runner catches every `Exception` in `_run_once`, so any of the
above becomes a `pipeline_error` log line and the retry budget
decides whether to give up.
# Wiring
```
_run_once(config) # main.py
├── ensure_repo(...)
├── store = PostedStore(config.data_dir)
├── candidates = list(_iter_candidates(config)) # find_anniversary_matches
├── unposted = [m for m in candidates if not store.is_posted(m.path)]
├── if unposted:
│ publish_mastodon(config, unposted) # <-- THIS boundary
│ store.mark_posted_many(posted_ids) # only after success
```
State is written **only after** `publish_mastodon` returns. A publish
failure leaves `posted.json` unchanged and lets the retry budget
re-attempt on the next iteration.
# Configuration Surface
| Env var | Consumed via | Effect |
|---|---|---|
| `MASTODON_BASE_URL` | `Config.mastodon_base_url` | Mastodon instance URL. |
| `MASTODON_ACCESS_TOKEN` | `Config.mastodon_access_token` | OAuth token passed to the `Mastodon` client. |
| `VISIBILITY` | `Config.visibility` | Passed as `visibility=` to `status_post`. |
| `THROWNBACK_PREFIX` | `Config.throwback_prefix` | First line of every published status. |
| `HASHTAGS` | `Config.hashtags` | Trailing hashtag line; commas become spaces. |
See [Config Schema](/architecture/config-schema.md) for full validation
rules.
# Key Files
| Path | Responsibility |
|---|---|
| `/repo/src/tenbackward/publishing.py` | `MASTODON_STATUS_LIMIT`, `PublishError`, `build_status_text`, `validate_status`, `_post_status_via_mastodon_py`, `publish_mastodon`. |
| `/repo/src/tenbackward/matching.py` | `MatchedPost` dataclass and `slugify_title()` reused here. |
| `/repo/src/tenbackward/main.py` | Calls `publish_mastodon()` between dedupe and `mark_posted_many`. |
| `/repo/tests/test_publishing.py` | Composition, length validation, visibility passthrough, API failure wrapping, and the no-persistence-on-failure contract. |
# Related
* [Pipeline Runner](/architecture/pipeline-runner.md)
* [Anniversary Matching](/architecture/anniversary-matching.md)
* [Config Schema](/architecture/config-schema.md)
* [Daily Run Guide](/guides/daily-run.md)
docs/architecture/pipeline-runner.md
+26 -17
View File
@@ -10,9 +10,8 @@ timestamp: 2026-08-04T17:51:00Z
`tenbackward.main` is the entry point executed by cron. The pipeline first
ensures that the configured blog repository is available and current, then
scans Jekyll posts for today's ten-year anniversary, filters already-recorded
paths, and persists newly discovered identifiers. Job 1086 wires the
anniversary matcher into the runner; Mastodon publishing remains a future
step.
paths, publishes a single combined Mastodon status for the new matches, and
finally persists the freshly published identifiers.
# Call Flow
@@ -28,8 +27,9 @@ main.main()
├── result = _run_with_retry(config)
│ ├── for attempt in 1 .. max_retries+1:
│ │ try: return _run_once(config)
│ │ └── _run_once → ensure_repo → iter_anniversary_paths
│ │ → PostedStore deduplication → mark_posted_many
│ │ └── _run_once → ensure_repo → find_anniversary_matches
│ │ → PostedStore deduplication → publish_mastodon
│ │ → mark_posted_many # state only after publish OK
│ ├── log_error("pipeline_error", exc=exc, attempt=attempt, max_attempts=attempts)
│ ├── log_error("pipeline_failed", exc=last_exc, attempts=attempts)
│ └── return None
@@ -40,12 +40,16 @@ main.main()
# Extension Point: `_iter_candidates`
```python
def _iter_candidates(config: Config) -> Iterable[str]: ...
def _iter_candidates(config: Config) -> Iterable[MatchedPost]: ...
```
Job 1086 wires `_iter_candidates` to the anniversary matcher. The matcher
is now active; it discovers matching Jekyll posts after the repository is
synchronized.
`_iter_candidates` delegates to `find_anniversary_matches()` with
`config.blog_dir / "_posts" / "blog"` and `config.site_url`. The matcher
returns full `MatchedPost` values (relative path, title, date, canonical
URL); the runner uses `MatchedPost.path` as a stable deduplication ID and
passes the full objects to `publish_mastodon()` for status composition.
See [Anniversary Matching](/architecture/anniversary-matching.md) for the
file and front matter rules.
## Blog Synchronization
@@ -56,14 +60,6 @@ fail immediately with `BlogRepoError`. The same synchronization occurs once
in `main()` before `startup` and again inside `_run_once()` as the retryable
pipeline boundary.
`_iter_candidates(config)` delegates to `iter_anniversary_paths()` with
`config.blog_dir / "_posts" / "blog"` and `config.site_url`. The matcher
returns relative Markdown paths, such as
`2016/2016-08-04-example.md`, which are used as stable deduplication IDs.
See [Anniversary Matching](/architecture/anniversary-matching.md) for the
file and front matter rules.
# `_run_once` — Summary Counters
`_run_once` returns a 5-tuple:
@@ -86,6 +82,19 @@ The store persists the list under a `"posted"` key
(`{"posted": ["2014/2014-08-04-foo.md", ...]}`) and serialises
concurrent runs with an `fcntl.flock`.
# Publish Boundary
`publish_mastodon()` is the single success boundary. The runner calls it
**after** dedupe and **before** `mark_posted_many`. If the call raises a
`PublishError`, the runner treats it like any other pipeline exception —
the retry loop in `_run_with_retry` re-attempts, and no identifier is
written to `posted.json`. Identical-day matches are published as **one**
combined status: prefix line + one `title\nurl` block per post + hashtags
line, sorted by `(date, path)` for deterministic ordering. The default
prefix is `Heute vor 10 Jahren:` (overridable via `THROWNBACK_PREFIX`).
Generated statuses are validated against `MASTODON_STATUS_LIMIT` (500
characters); the runner fails safely (raises) rather than truncating.
# Retry Behaviour (`_run_with_retry`)
* `attempts = max(1, config.max_retries + 1)` — at least one attempt
docs/architecture/system-overview.md
+31 -7
View File
@@ -3,7 +3,7 @@ type: architecture
title: System Architecture
description: Component map of 10Backward — how configuration, blog synchronization, anniversary matching, logging, the pipeline runner, state persistence, and container entrypoint are wired together.
tags: [architecture, overview]
timestamp: 2026-08-04T17:51:00Z
timestamp: 2026-08-04T18:55:00Z
---
# Overview
@@ -12,8 +12,9 @@ timestamp: 2026-08-04T17:51:00Z
containerised cron job. Job 1086 adds anniversary matching across Jekyll
posts, backed by the blog clone/pull workflow from Job 1084 and the
structured logging, retry-capable runner, and deduplication state from
Job 1083. The current pipeline identifies matching post paths and records
those identifiers; Mastodon publishing is still a future integration.
Job 1083. The current pipeline identifies matching posts, publishes a
single combined Mastodon status for the new matches, and records the
published identifiers in `posted.json`.
The container boots, validates environment configuration, renders a
`/etc/cron.d/tenbackward` entry that fires once per day at the
@@ -25,7 +26,9 @@ foreground of the `cron` process. Each scheduled invocation calls
2. Loads and validates the `Config`.
3. Ensures the data directory exists.
4. Emits a `startup` log line.
5. Executes one pipeline pass wrapped in a retry loop.
5. Executes one pipeline pass wrapped in a retry loop:
`ensure_repo` → `find_anniversary_matches` → `PostedStore` dedupe →
`publish_mastodon` → `PostedStore.mark_posted_many`.
6. Emits a `run_complete` summary (skipped silently if there were no candidates).
# Components
@@ -39,6 +42,7 @@ foreground of the `cron` process. Each scheduled invocation calls
| `tenbackward.state` | `PostedStore` — self-contained dedup store. `posted.json` holds `{"posted": [str, ...]}`; mutations are serialised by an `fcntl.flock` on a sibling lock file, writes go through a temp-file replace, and `load()` auto-creates an empty list when the file is missing. |
| `tenbackward.blog` | GitPython `ensure_repo()` — clones the configured blog repo on first run, fast-forwards it via `pull --ff-only` thereafter, with exponential-backoff retries on transient network errors. |
| `tenbackward.matching` | Walks `_posts/blog/**/*.md`, parses Jekyll front matter and filenames, and yields posts exactly ten years before the current date using Berlin-time and leap-day rules. |
| `tenbackward.publishing` | `publish_mastodon()` — the single success boundary between the runner and the Mastodon HTTP API. Composes a combined status for the day's matches, validates the 500-character limit, and posts via `mastodon.Mastodon.status_post`. Raises `PublishError` on composition, length, or API failure; the runner treats it like any other pipeline exception. |
| `/etc/cron.d/tenbackward`| Rendered cron file. One daily line that `cd /app` and runs `python -m tenbackward`. |
# Communication & Wiring
@@ -61,9 +65,23 @@ foreground of the `cron` process. Each scheduled invocation calls
+-------------------------+
| _run_with_retry(...) |
| -> _run_once(...) |
| -> PostedStore. |
| -> ensure_repo(...) |
| -> matching |
| find_anniversary_|
| matches |
| -> PostedStore |
| {is_posted, |
| mark_posted_many}|
| -> publishing |
| publish_mastodon |
| -> PostedStore |
| mark_posted_many |
+-------------------------+
|
v
+-------------------------+
| Mastodon HTTP API |
| status_post(...) |
+-------------------------+
```
@@ -73,10 +91,14 @@ foreground of the `cron` process. Each scheduled invocation calls
`Config` dataclass.
* **Cron → main.** Each scheduled tick re-runs `python -m tenbackward`,
so every run is a fresh interpreter invocation.
* **Matching → publishing.** `find_anniversary_matches` yields
`MatchedPost` objects; the runner dedupes against `PostedStore`, then
passes the unposted objects to `publish_mastodon`. See
[Mastodon Publishing](/architecture/mastodon-publishing.md).
* **Pipeline state.** `_run_once` reads `posted.json` via
`PostedStore.is_posted()` for dedupe, then writes it back via
`PostedStore.mark_posted_many()` only when at least one new post
was recorded. The store serialises concurrent runs with an
`PostedStore.mark_posted_many()` **only after** `publish_mastodon`
returns. The store serialises concurrent runs with an
`fcntl.flock` on a sibling lock file and writes via temp-file
rename.
@@ -91,6 +113,7 @@ foreground of the `cron` process. Each scheduled invocation calls
| `/repo/src/tenbackward/state.py` | `PostedStore` + module helpers (`load_posted`, `is_posted`, `mark_posted`, `mark_posted_many`); list-shaped JSON, `fcntl.flock`, atomic temp-file replace, env-var data dir. |
| `/repo/src/tenbackward/blog.py` | `ensure_repo()` — GitPython clone + fast-forward pull with `2^n` retry/backoff; raises `BlogRepoError` on local modifications or exhausted retries. |
| `/repo/src/tenbackward/matching.py` | Jekyll post discovery, front matter parsing, anniversary and leap-day matching, and canonical URL construction. |
| `/repo/src/tenbackward/publishing.py` | `publish_mastodon()` — composes a combined Mastodon status, validates the 500-character limit, posts via `mastodon.Mastodon.status_post`. Pure helpers `build_status_text` and `validate_status` keep network code out of tests. |
| `/repo/src/tenbackward/__main__.py` | Module entrypoint that invokes `tenbackward.main.main()`. |
| `/repo/.env.example` | Canonical list of environment variables. |
| `/repo/docker-compose.yml` | Service definition; binds env vars from the host `.env`. |
@@ -105,5 +128,6 @@ foreground of the `cron` process. Each scheduled invocation calls
* [Logging & Run Summary](/architecture/logging.md)
* [Pipeline Runner](/architecture/pipeline-runner.md)
* [Anniversary Matching](/architecture/anniversary-matching.md)
* [Mastodon Publishing](/architecture/mastodon-publishing.md)
* [Environment Variable Setup](/operations/environment-setup.md)
* [Cron Lifecycle](/operations/cron-lifecycle.md)
docs/guides/daily-run.md
+7 -4
View File
@@ -29,9 +29,12 @@ scheduled tick.
Before scanning, the process ensures the configured blog repository exists
under `/app/data/blog` and is fast-forwarded from `BLOG_REPO_URL`. It then
walks `_posts/blog/**/*.md`, finds posts whose date is exactly ten years
before today, skips IDs already present in `posted.json`, and records new
relative paths. The current implementation records matching IDs only;
Mastodon publication is not yet wired.
before today, skips IDs already present in `posted.json`, composes a
single German-language Mastodon status (default prefix
`Heute vor 10 Jahren:`) listing each new match's title and canonical URL
followed by the configured hashtags, and — only after the Mastodon API
call succeeds — records the relative paths of the published posts in
`posted.json`.
# What You Should See
@@ -42,7 +45,7 @@ The bot uses a **JSON-per-line** logger that writes to
should show output similar to:
```json
{"ts": "2026-08-04T09:00:00+00:00", "level": "INFO", "logger": "tenbackward", "message": "startup", "event": "startup", "version": "0.1.0", "site_url": "https://blog.example.com", "run_at": "09:00", "tz": "Europe/Berlin", "hashtags": "#throwback,#10backward", "throwback_prefix": "Throwback:"}
{"ts": "2026-08-04T09:00:00+00:00", "level": "INFO", "logger": "tenbackward", "message": "startup", "event": "startup", "version": "0.1.0", "site_url": "https://blog.example.com", "run_at": "09:00", "tz": "Europe/Berlin", "hashtags": "#throwback,#10backward", "throwback_prefix": "Heute vor 10 Jahren:"}
```
On the current scaffold (`_iter_candidates` is intentionally empty),
docs/index.md
+4 -3
View File
@@ -3,15 +3,16 @@ okf_version: "0.1"
---
# Architecture
* [System Architecture](/architecture/system-overview.md) — Component map of 10Backward: entrypoint, config, logging, runner, state, and cron wiring.
* [System Architecture](/architecture/system-overview.md) — Component map of 10Backward: entrypoint, config, logging, runner, state, publishing boundary, and cron wiring.
* [Config Schema](/architecture/config-schema.md) — Required/optional env vars, validation rules, and the typed Config dataclass.
* [Logging & Run Summary](/architecture/logging.md) — JSON formatter, secret redaction, and structured event helpers.
* [Pipeline Runner](/architecture/pipeline-runner.md) — How a cron tick synchronizes the blog, matches anniversaries, applies deduplication, retries failures, and emits run events.
* [Pipeline Runner](/architecture/pipeline-runner.md) — How a cron tick synchronizes the blog, matches anniversaries, applies deduplication, publishes to Mastodon, retries failures, and emits run events.
* [Anniversary Matching](/architecture/anniversary-matching.md) — Jekyll filename/front matter rules, ten-year date matching, leap-day handling, and candidate identifiers.
* [Mastodon Publishing](/architecture/mastodon-publishing.md) — The single success boundary: status composition, 500-character validation, and the Mastodon API call.
# Operations
* [Environment Variable Setup](/operations/environment-setup.md) — Required env vars, renamed keys, and how to populate `.env`.
* [Cron Lifecycle](/operations/cron-lifecycle.md) — How `entrypoint.sh` renders `/etc/cron.d/tenbackward` and hands off to `cron -f`.
# User Guides
* [Daily Run Guide](/guides/daily-run.md) — Tester/operator walkthrough of repository sync, anniversary matching, deduplication, and expected log/output behaviour.
* [Daily Run Guide](/guides/daily-run.md) — Tester/operator walkthrough of repository sync, anniversary matching, deduplication, Mastodon publishing, and expected log/output behaviour.
docs/operations/environment-setup.md
+1 -1
View File
@@ -21,7 +21,7 @@ full set before the container will boot.
| `VISIBILITY` | `public` | Post visibility (`public` or `unlisted`). |
| `SITE_URL` | `https://blog.example.com` | Source blog URL (used by the future clone step). |
| `HASHTAGS` | `#throwback,#10backward` | Hashtags appended to every throwback post. |
| `THROWNBACK_PREFIX` | `Throwback:` | Prefix prepended to every post. |
| `THROWNBACK_PREFIX` | `Heute vor 10 Jahren:` | Prefix prepended to every Mastodon status. |
| `MAX_RETRIES` | `3` | Non-negative retry count for the pipeline. |
| `RUN_AT` | `09:00` | Daily fire time (HH:MM, 24-hour). |
| `TZ` | `Europe/Berlin` | IANA timezone for cron + container clock. |