AI Implementation feature(1083): Environment Configuration and Logging Baseline (#2)

2026-08-04 17:53:19 +00:00
parent 33c6c76ce9
commit 207e2e6bbb
21 changed files with 1429 additions and 77 deletions
@@ -0,0 +1,111 @@
+---
+type: architecture
+title: Pipeline Runner
+description: How tenbackward.main turns one cron tick into a startup log line, a retry-wrapped pipeline pass, and a single run_complete summary line.
+tags: [pipeline, runner, retry]
+timestamp: 2026-08-04T17:51:00Z
+---
+
+# Purpose
+
+`tenbackward.main` is the entry point executed by cron. Job 1083
+rewrote it to: (a) install JSON logging, (b) execute a retry-wrapped
+pipeline pass that is **silent when there are no candidates**, and
+(c) emit a single structured `run_complete` summary on success.
+
+# Call Flow
+
+```
+main.main()
+  ├── configure_json_logging()
+  ├── load_config()  ── raises ConfigError → exit 2
+  ├── config.data_dir.mkdir(parents=True, exist_ok=True)
+  ├── log_startup(version, site_url, run_at, tz, hashtags, throwback_prefix)
+  ├── result = _run_with_retry(config)
+  │       ├── for attempt in 1 .. max_retries+1:
+  │       │     try: return _run_once(config)
+  │       │     except Exception as exc:
+  │       │           log_error("pipeline_error", exc=exc, attempt=attempt, max_attempts=attempts)
+  │       ├── log_error("pipeline_failed", exc=last_exc, attempts=attempts)
+  │       └── return None
+  ├── if result is None: return 1
+  └── log_run_summary(scanned, matched, posted, skipped, posted_ids)
+```
+
+# Extension Point: `_iter_candidates`
+
+```python
+def _iter_candidates(config: Config) -> Iterable[str]: ...
+```
+
+Job 1083 ships `_iter_candidates` as an **empty iterator** so the
+silent-on-no-matches contract is the default behaviour. The future
+blog-clone + matching pipeline plugs into this function without
+touching `_run_once` or `_run_with_retry`.
+
+# `_run_once` — Summary Counters
+
+`_run_once` returns a 5-tuple:
+
+```python
+(scanned, matched, posted, skipped, posted_ids)
+```
+
+| Counter     | Meaning                                                              |
+|-------------|----------------------------------------------------------------------|
+| `scanned`   | Items yielded by `_iter_candidates`.                                 |
+| `matched`   | Items that survived the dedupe-vs-`already_posted` check.            |
+| `posted`    | Items newly recorded in `posted.json` during this run.               |
+| `skipped`   | Items that matched but were already in `posted.json` (deduped).      |
+| `posted_ids`| The list of post identifiers written to state.                       |
+
+`state.save_posted` is called **only** when `posted_ids` is non-empty,
+so a no-op run does not touch `posted.json` on disk.
+
+# Retry Behaviour (`_run_with_retry`)
+
+* `attempts = max(1, config.max_retries + 1)` — at least one attempt
+  even if `MAX_RETRIES=0`.
+* Any exception inside `_run_once` is caught (`noqa: BLE001` —
+  intentional, the whole pass is opaque to the runner).
+* Each failed attempt is recorded via
+  `log_error("pipeline_error", exc=exc, attempt=attempt, max_attempts=attempts)`.
+* When the budget is exhausted, the runner emits
+  `log_error("pipeline_failed", exc=last_exc, attempts=attempts)` and
+  returns `None` so `main` can translate it to `exit 1`.
+* `time.sleep(0)` between attempts is the placeholder seam where the
+  next job can introduce real backoff.
+
+# Silent-on-No-Matches Contract
+
+A run with zero candidates produces exactly one line:
+
+```json
+{"ts":"...","level":"INFO","message":"startup","event":"startup",...}
+```
+
+No `run_complete` line is emitted. Tests assert this by counting log
+records (`tests/test_run_logging.py::test_run_summary_silent_when_no_candidates`).
+
+# Example Sequence (Healthy Run)
+
+1.  `startup` line at INFO.
+2.  `_run_with_retry` returns `(0, 0, 0, 0, [])` (no candidates yet
+    because `_iter_candidates` is empty) → `main` exits `0`
+    **without** a `run_complete` line.
+3.  Next-day tick repeats the cycle.
+
+# Example Sequence (Future Blog Step in Place)
+
+1.  `startup` line at INFO.
+2.  `_iter_candidates` yields `[id-1, id-2]`. `id-1` is already in
+    `posted.json`, so `skipped += 1`. `id-2` is new and appended to
+    `posted_ids` and persisted.
+3.  `run_complete` line at INFO with
+    `scanned=2, matched=2, posted=1, skipped=1, posted_ids=["id-2"]`.
+
+# Related
+
+* [Logging & Run Summary](/architecture/logging.md)
+* [Config Schema](/architecture/config-schema.md)
+* [Cron Lifecycle](/operations/cron-lifecycle.md)