AI Implementation feature(1086): Posted-Anniversary Dedup Store (#5)

docs/architecture/pipeline-runner.md

@@ -1,17 +1,19 @@
 ---
 type: architecture
 title: Pipeline Runner
-description: How tenbackward.main turns one cron tick into a startup log line, a retry-wrapped pipeline pass, and a single run_complete summary line.
+description: How tenbackward.main synchronizes the blog, discovers anniversary candidates, applies deduplication, retries failures, and emits a run summary.
 tags: [pipeline, runner, retry]
 timestamp: 2026-08-04T17:51:00Z
 ---
 
 # Purpose
+`tenbackward.main` is the entry point executed by cron. The pipeline first
+ensures that the configured blog repository is available and current, then
+scans Jekyll posts for today's ten-year anniversary, filters already-recorded
+paths, and persists newly discovered identifiers. Job 1086 wires the
+anniversary matcher into the runner; Mastodon publishing remains a future
+step.
 
-`tenbackward.main` is the entry point executed by cron. Job 1083
-rewrote it to: (a) install JSON logging, (b) execute a retry-wrapped
-pipeline pass that is **silent when there are no candidates**, and
-(c) emit a single structured `run_complete` summary on success.
 
 # Call Flow
 
@@ -20,12 +22,15 @@ main.main()
   ├── configure_json_logging()
   ├── load_config()  ── raises ConfigError → exit 2
   ├── config.data_dir.mkdir(parents=True, exist_ok=True)
+  ├── ensure_repo(config.data_dir, blog_repo_url, blog_dir, max_retries)
+  │       └── BlogRepoError → log blog_repo_error → exit 1
   ├── log_startup(version, site_url, run_at, tz, hashtags, throwback_prefix)
   ├── result = _run_with_retry(config)
   │       ├── for attempt in 1 .. max_retries+1:
   │       │     try: return _run_once(config)
-  │       │     except Exception as exc:
-  │       │           log_error("pipeline_error", exc=exc, attempt=attempt, max_attempts=attempts)
+  │       │     └── _run_once → ensure_repo → iter_anniversary_paths
+  │       │         → PostedStore deduplication → mark_posted_many
+  │       ├── log_error("pipeline_error", exc=exc, attempt=attempt, max_attempts=attempts)
   │       ├── log_error("pipeline_failed", exc=last_exc, attempts=attempts)
   │       └── return None
   ├── if result is None: return 1
@@ -38,10 +43,26 @@ main.main()
 def _iter_candidates(config: Config) -> Iterable[str]: ...
 ```
 
-Job 1083 ships `_iter_candidates` as an **empty iterator** so the
-silent-on-no-matches contract is the default behaviour. The future
-blog-clone + matching pipeline plugs into this function without
-touching `_run_once` or `_run_with_retry`.
+Job 1086 wires `_iter_candidates` to the anniversary matcher. The matcher
+is now active; it discovers matching Jekyll posts after the repository is
+synchronized.
+
+## Blog Synchronization
+
+`ensure_repo()` runs before candidate discovery. An empty target is cloned
+from `BLOG_REPO_URL`; an existing repository is fetched and fast-forwarded.
+Transient Git failures use exponential backoff, while local modifications
+fail immediately with `BlogRepoError`. The same synchronization occurs once
+in `main()` before `startup` and again inside `_run_once()` as the retryable
+pipeline boundary.
+
+
+`_iter_candidates(config)` delegates to `iter_anniversary_paths()` with
+`config.blog_dir / "_posts" / "blog"` and `config.site_url`. The matcher
+returns relative Markdown paths, such as
+`2016/2016-08-04-example.md`, which are used as stable deduplication IDs.
+See [Anniversary Matching](/architecture/anniversary-matching.md) for the
+file and front matter rules.
 
 # `_run_once` — Summary Counters
 
@@ -59,8 +80,11 @@ touching `_run_once` or `_run_with_retry`.
 | `skipped`   | Items that matched but were already in `posted.json` (deduped).      |
 | `posted_ids`| The list of post identifiers written to state.                       |
 
-`state.save_posted` is called **only** when `posted_ids` is non-empty,
-so a no-op run does not touch `posted.json` on disk.
+`PostedStore.mark_posted_many` is called **only** when `posted_ids`
+is non-empty, so a no-op run does not touch `posted.json` on disk.
+The store persists the list under a `"posted"` key
+(`{"posted": ["2014/2014-08-04-foo.md", ...]}`) and serialises
+concurrent runs with an `fcntl.flock`.
 
 # Retry Behaviour (`_run_with_retry`)
 
@@ -73,19 +97,11 @@ so a no-op run does not touch `posted.json` on disk.
 * When the budget is exhausted, the runner emits
   `log_error("pipeline_failed", exc=last_exc, attempts=attempts)` and
   returns `None` so `main` can translate it to `exit 1`.
-* `time.sleep(0)` between attempts is the placeholder seam where the
-  next job can introduce real backoff.
 
 # Silent-on-No-Matches Contract
 
-A run with zero candidates produces exactly one line:
-
-```json
-{"ts":"...","level":"INFO","message":"startup","event":"startup",...}
-```
-
-No `run_complete` line is emitted. Tests assert this by counting log
-records (`tests/test_run_logging.py::test_run_summary_silent_when_no_candidates`).
+* `run_complete` is emitted for a successful pass, including when all discovered candidates were already posted.
+* A run with zero candidates emits only `startup`; no `run_complete` line is emitted.
 
 # Example Sequence (Healthy Run)
 
@@ -106,6 +122,7 @@ records (`tests/test_run_logging.py::test_run_summary_silent_when_no_candidates`
 
 # Related
 
+* [Anniversary Matching](/architecture/anniversary-matching.md)
 * [Logging & Run Summary](/architecture/logging.md)
 * [Config Schema](/architecture/config-schema.md)
 * [Cron Lifecycle](/operations/cron-lifecycle.md)