Skip to content

answers-recheck: fresh live samples re-checked weekly, drifted answers held back from the pack - #7

Merged
SecureCloudGroup merged 1 commit into
mainfrom
round19/phase2a
Oct 7, 2026
Merged

SecureCloudGroup merged 1 commit into
mainfrom
round19/phase2a

Conversation

@SecureCloudGroup

Copy link
Copy Markdown
Owner

Round 19 execution plan, Phase 2a: the part that needs no ruling. The Library gains the drift detection the authoring loop was missing: until now checked was never updated after an answers file was created, and nothing noticed when a source's response changed.

What changes

  • sourcetool answers-recheck [--since DAYS] [ID ...] re-fetches each answered source's live sample (cache bypassed) and runs the same check as answers-check, then writes the verdict into the file. PASS stamps checked with today's date and drops any failure keys. FAIL is remembered in last_failure. A second failure on a later day marks the file status: "drifted"; build holds it back and the app offers the source as a link until a later pass recovers it. Each file keeps its own indent and final newline, so a recheck diff is one line.
  • Transients are not drift evidence. A connection-level failure is retried once; an HTTP status, a robots refusal or a parse failure is the source's own answer and is not retried.
  • A missing app checkout is a SKIP, never a FAIL. fetch_sample borrows the app's csv / feed / xml / text parsers from a sibling SmartBrain_3000 checkout. Without one, every command now skips those sources instead of failing them. Before this fix the weekly job, which has no such checkout, would have drifted all 87 non-JSON sources within two weeks.
  • Weekly job (revalidate, Mondays): re-validates curated records as before, then re-checks every answers file with the app's parsers checked out sparsely beside the Library, pushes any changed files to recheck/<date> for a PR you open, writes the log to the step summary, and turns red when a file failed or drifted. The job gains contents: write for that push only.
  • Spec v1.3 keys status and last_failure, written only by the recheck, validated by load_answers; drifted files are excluded from build and the lints but still seen by the live check. SCHEMA.md documents the rule.

Verified

  • sourcetool check clean; answers-check --lint 0 problems; 255 tests (7 new: state transitions, formatting preserved, --since, held-back files, key validation, parser skip, retry).
  • Live smoke in the dev image: nws-alerts-point, open-meteo-forecast (JSON), fred-apu0000702111 (CSV), npr-news, bbc-world (feeds) all PASS; their checked dates in this PR come from those runs.

Not in this PR (waits on the plan's rulings)

answers-draft (decision 2: which model drafts), expects for the first subcategories (decision 3), the gallery and publish verdict (needs the app's forms from Phase 1a, decision 1).

…eld back from the pack

`sourcetool answers-recheck [--since DAYS] [ID ...]` re-fetches each answered source's live sample (cache
bypassed), runs the same check as answers-check, and writes the result into the answers file: a pass stamps
`checked` with today's date and drops any failure keys; a failure is remembered in `last_failure`; a second
failure on a later day marks the file `status: drifted`, which `build` holds back so the app offers the
source as a link until a later pass recovers it. A connection-level failure is retried once (a transient
is not drift evidence); an HTTP status, a robots refusal or a parse failure is the source's own answer. A
missing SmartBrain_3000 checkout (the csv / feed / xml / text parsers) is now a SKIP for every command, never
a FAIL — before this the weekly job would have drifted every non-JSON source within two weeks. Each file
keeps its own formatting, so a recheck diff is one line.

The weekly job re-checks every file with the app's parsers checked out beside the Library, pushes any
changed answers files to recheck/<date> for a PR, and turns red when a file failed or drifted. The live
check (answers-check) still exercises drifted files so a recovery can be seen on demand. Five answers files
carry today's `checked` from the smoke runs (JSON, CSV and feed sources all pass).

Round 19 execution plan, Phase 2a (decision-free part).
@SecureCloudGroup
SecureCloudGroup merged commit f1fa9b7 into main Oct 7, 2026
3 checks passed
@SecureCloudGroup
SecureCloudGroup deleted the round19/phase2a branch October 7, 2026 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant