m0rph3us1987/10Backward
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

AI Implementation feature(1089): Scheduled Execution with Cron and Retry Handling (#9)

Browse Source
This commit was merged in pull request #9.
This commit is contained in:
m0rph3us1987
2026-08-04 20:48:16 +00:00
parent 3545b81546
commit 5f6ddf76a4
27 changed files with 644 additions and 272 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/config-schema.md
+23 -35
View File
@@ -16,45 +16,33 @@ dataclass. Job 1083 changes every variable that ships in
# Required Environment Variables
All eight of these must be present and non-empty in the process
environment for the container to boot.
The following keys must be present and non-empty in the merged dotenv/process environment before application defaults are applied.
| Key | Purpose | Validation |
|----------------------|--------------------------------------------------------|------------------------------------------------------------------|
| `MASTODON_BASE_URL` | Mastodon instance to post against. | Must parse with `http` or `https` scheme and a non-empty netloc. |
| `MASTODON_ACCESS_TOKEN` | OAuth token used by the future Mastodon.py client. | Required string. |
| `VISIBILITY` | Default post visibility. | Must be `public` or `unlisted`. |
| `SITE_URL` | Source blog URL (used by the future clone step). | Must parse with `http` or `https` scheme and a non-empty netloc. |
| `HASHTAGS` | Comma-separated hashtags appended to every throwback. | Required string (formatting handled by the future poster). |
| `THROWNBACK_PREFIX` | Per-post title prefix (default: `Throwback:`). | Required string. |
| `MAX_RETRIES` | Number of additional retries after the first attempt. | Non-negative integer (parsed by `_parse_max_retries`). |
| `RUN_AT` | Daily fire time for the cron entry. | `HH:MM` 24-hour format. |
| `TZ` | IANA timezone used by cron + the container clock. | Must resolve via `zoneinfo.ZoneInfo`. |
| Key | Purpose | Validation |
|---|---|---|
| `MASTODON_BASE_URL` | Mastodon instance to post against. | Valid `http` or `https` URL. |
| `MASTODON_ACCESS_TOKEN` | OAuth token used by the Mastodon client. | Required string; redacted in logs. |
| `MASTODON_VISIBILITY` | Post visibility. | `public` or `unlisted`. |
| `SITE_URL` | Source blog URL. | Valid `http` or `https` URL. |
| `HASHTAGS` | Comma-separated hashtags. | Required string. |
| `THROWBACK_PREFIX` | Prefix prepended to each status. | Required string. |
| `MAX_RETRIES` | Maximum number of pipeline attempts. | Non-negative integer; defaults to `5` when absent. |
| `RUN_AT` | Daily cron fire time. | `HH:MM` 24-hour format. |
| `TZ` | Cron and application timezone. | Valid IANA timezone. |
> **Note** — All nine keys are required since Job 1083. The previously
> optional `MASTODON_VISIBILITY`, `THROWBACK_PREFIX`, and `RETRY_COUNT`
> were promoted to first-class citizens and renamed to `VISIBILITY`,
> `THROWNBACK_PREFIX`, and `MAX_RETRIES`.
`BLOG_REPO_URL` and `BLOG_DIR` are optional and receive defaults after this required-key check. `DATA_DIR` is read separately from the process environment and defaults to `/app/data`.
# Defaults
`DEFAULTS` in `config.py` provides fallback strings that `apply_defaults`
fills into the merged map **before** validation runs:
Defaults are applied only after required-key validation. The current defaults are:
| Default key | Default value |
|-----------------------|-------------------------------------------|
| `VISIBILITY` | `public` |
| `THROWNBACK_PREFIX` | `Heute vor 10 Jahren:` |
| `MAX_RETRIES` | `3` |
| `TZ` | `Europe/Berlin` |
| `RUN_AT` | `09:00` |
| `BLOG_REPO_URL` | `https://git.chaospott.de/Chaospott/site` |
| `BLOG_DIR` | `blog` (resolved relative to `data_dir`) |
| Default key | Default value |
|---|---|
| `MAX_RETRIES` | `5` |
| `BLOG_REPO_URL` | `https://git.chaospott.de/Chaospott/site` |
| `BLOG_DIR` | `blog` (resolved relative to `data_dir`) |
Because every required key carries a default, a freshly-initialised
container can still boot to validate the scaffold, but the operator
must supply `MASTODON_BASE_URL` and `MASTODON_ACCESS_TOKEN` (and
ideally `SITE_URL`) to make a real run.
Application-facing required values are not silently supplied by `config.py`; the container environment may still provide `TZ=Europe/Berlin` and Compose may provide development fallback values.
# `Config` Dataclass
@@ -62,14 +50,14 @@ ideally `SITE_URL`) to make a real run.
|-------------------------|-----------|----------------------------------------------|
| `mastodon_base_url` | `str` | `MASTODON_BASE_URL` |
| `mastodon_access_token` | `str` | `MASTODON_ACCESS_TOKEN` |
| `visibility` | `str` | `VISIBILITY` |
| `mastodon_visibility` | `str` | `MASTODON_VISIBILITY` |
| `site_url` | `str` | `SITE_URL` |
| `hashtags` | `str` | `HASHTAGS` |
| `throwback_prefix` | `str` | `THROWNBACK_PREFIX` |
| `throwback_prefix` | `str` | `THROWBACK_PREFIX` |
| `max_retries` | `int` | `MAX_RETRIES` (parsed as non-negative int) |
| `run_at` | `str` | `RUN_AT` |
| `tz` | `str` | `TZ` |
| `data_dir` | `Path` | Constant: `Path("/app/data")` |
| `data_dir` | `Path` | `DATA_DIR`, default `/app/data` |
| `blog_repo_url` | `str` | `BLOG_REPO_URL` |
| `blog_dir` | `Path` | `BLOG_DIR` resolved against `data_dir` |
docs/architecture/mastodon-publishing.md
+4 -8
View File
@@ -45,8 +45,7 @@ of matching posts, in this shape:
{hashtags}
```
* The prefix comes from `Config.throwback_prefix` (default
`Heute vor 10 Jahren:`, overridable via `THROWNBACK_PREFIX`).
* The prefix comes from `Config.throwback_prefix` (configured via `THROWBACK_PREFIX`).
* Posts are sorted by `(date, path)` so output is deterministic
regardless of upstream order.
* `hashtags` is a comma-separated string; commas are collapsed to a
@@ -106,10 +105,7 @@ publish failure.
# Mastodon API Call
`_post_status_via_mastodon_py()` constructs a `mastodon.Mastodon`
client with `(access_token=config.mastodon_access_token,
api_base_url=config.mastodon_base_url)` and calls
`status_post(status, visibility=config.visibility)`.
`_post_status_via_mastodon_py()` constructs a `mastodon.Mastodon` client with `(access_token=config.mastodon_access_token, api_base_url=config.mastodon_base_url)` and calls `status_post(status, visibility=config.mastodon_visibility)`.
The `client_factory` keyword argument on `publish_mastodon` lets
tests inject a fake client without monkey-patching. Production
@@ -150,8 +146,8 @@ re-attempt on the next iteration.
|---|---|---|
| `MASTODON_BASE_URL` | `Config.mastodon_base_url` | Mastodon instance URL. |
| `MASTODON_ACCESS_TOKEN` | `Config.mastodon_access_token` | OAuth token passed to the `Mastodon` client. |
| `VISIBILITY` | `Config.visibility` | Passed as `visibility=` to `status_post`. |
| `THROWNBACK_PREFIX` | `Config.throwback_prefix` | First line of every published status. |
| `MASTODON_VISIBILITY` | `Config.mastodon_visibility` | Passed as `visibility=` to `status_post`. |
| `THROWBACK_PREFIX` | `Config.throwback_prefix` | First line of every published status. |
| `HASHTAGS` | `Config.hashtags` | Trailing hashtag line; commas become spaces. |
See [Config Schema](/architecture/config-schema.md) for full validation
docs/architecture/pipeline-runner.md
+6 -9
View File
@@ -97,15 +97,12 @@ characters); the runner fails safely (raises) rather than truncating.
# Retry Behaviour (`_run_with_retry`)
* `attempts = max(1, config.max_retries + 1)` — at least one attempt
even if `MAX_RETRIES=0`.
* Any exception inside `_run_once` is caught (`noqa: BLE001` —
intentional, the whole pass is opaque to the runner).
* Each failed attempt is recorded via
`log_error("pipeline_error", exc=exc, attempt=attempt, max_attempts=attempts)`.
* When the budget is exhausted, the runner emits
`log_error("pipeline_failed", exc=last_exc, attempts=attempts)` and
returns `None` so `main` can translate it to `exit 1`.
* `attempts = max(1, config.max_retries)` — at least one attempt even if `MAX_RETRIES=0`.
* The runner owns the unified retry budget; `_run_once` calls `ensure_repo` with `max_retries=0`.
* Transient Git and Mastodon failures are retried with delays of `2 ** (attempt - 1)` seconds between attempts.
* Each failed attempt emits `retry_attempt` with `operation`, `attempt`, and the exception `error` text.
* Fatal configuration, local repository, and publishing validation errors emit `retry_exhausted` immediately without retrying.
* When the budget is exhausted, `retry_exhausted` includes the final operation, total attempts, and error text, and `main` returns exit code `1`.
# Silent-on-No-Matches Contract
docs/architecture/runtime-changes.md
+66
View File
@@ -0,0 +1,66 @@
---
type: architecture
title: Job 1089 Runtime Changes
description: Container privilege separation, unified retry logging, configuration validation, and persistent data wiring introduced by the current implementation.
tags: [architecture, docker, retry, logging, configuration]
timestamp: 2026-08-04T20:44:00Z
---
# Overview
The current runtime keeps the container entrypoint and cron daemon as root while executing the Python bot as the unprivileged `bot` user. The pipeline owns one retry budget for Git synchronization and Mastodon publishing, and structured retry events include the operation and exception text. Configuration now validates required values before optional defaults are applied.
# Wiring
```text
Docker ENTRYPOINT /app/entrypoint.sh (root)
-> validates environment and renders /etc/cron.d/tenbackward
-> cron -f
-> /usr/local/bin/run-bot.sh
-> setpriv/su to bot
-> python -m tenbackward
-> load_config()
-> _run_with_retry()
-> _run_once()
-> ensure_repo(..., max_retries=0, logger=...)
-> match posts
-> PostedStore dedupe
-> publish_mastodon()
-> PostedStore.mark_posted_many()
```
The single `./data:/app/data` Compose bind mount contains both the blog clone and `posted.json`; no file-level bind mount is used.
# Retry and Logging Contract
`MAX_RETRIES` is interpreted as the maximum number of attempts, with at least one attempt when configured as zero. Backoff delays are `1, 2, 4, ...` seconds between attempts. Git clone/pull retries are surfaced to the runner with operation labels (`git clone` or `git pull`), while publishing failures use `mastodon post`. Fatal configuration, local-repository, and publishing validation errors are not retried.
Retry events include an `error` field containing the exception message. `log_event()` sends structured fields through the same redaction path as other log helpers.
# Configuration Validation
`load_config()` merges dotenv values and process environment values, validates required keys against that raw merged map, then applies defaults. `MAX_RETRIES`, `BLOG_REPO_URL`, and `BLOG_DIR` may use defaults; the required Mastodon, site, hashtag, prefix, schedule, and timezone values must be explicitly non-empty.
# Key Files
| Path | Responsibility |
|---|---|
| `/repo/Dockerfile` | Installs cron, Git, timezone data, and `setpriv`; leaves the final process root-capable for cron and owns `/app/data` by `bot`. |
| `/repo/entrypoint.sh` | Validates environment, renders the daily cron entry, installs it as root, and starts foreground cron. |
| `/repo/run-bot.sh` | Drops from root to `bot` using `setpriv` or `su`, then invokes the Python module. |
| `/repo/docker-compose.yml` | Supplies environment values and mounts `./data` at `/app/data`. |
| `/repo/src/tenbackward/main.py` | Owns the unified retry loop and passes logger/retry controls into repository synchronization. |
| `/repo/src/tenbackward/blog.py` | Clones or fast-forwards the blog and emits operation-specific retry events. |
| `/repo/src/tenbackward/config.py` | Performs raw required-key validation and builds the typed configuration. |
| `/repo/src/tenbackward/logging_setup.py` | Provides structured events, redaction, and JSON-per-line formatting. |
| `/repo/tests/test_docker_artifacts.py` | Verifies container privilege and artifact wiring. |
| `/repo/tests/test_config.py` | Verifies required-key validation and defaults. |
| `/repo/tests/test_run_logging.py` | Verifies retry fields, attempt behavior, and state safety. |
# Related
* [System Architecture](/architecture/system-overview.md)
* [Config Schema](/architecture/config-schema.md)
* [Pipeline Runner](/architecture/pipeline-runner.md)
* [Environment Variable Setup](/operations/environment-setup.md)
* [Daily Run Guide](/guides/daily-run.md)