Repository navigation
answers-recheck: fresh live samples re-checked weekly, drifted answers held back from the pack - #7
Merged
Merged
Conversation
…eld back from the pack `sourcetool answers-recheck [--since DAYS] [ID ...]` re-fetches each answered source's live sample (cache bypassed), runs the same check as answers-check, and writes the result into the answers file: a pass stamps `checked` with today's date and drops any failure keys; a failure is remembered in `last_failure`; a second failure on a later day marks the file `status: drifted`, which `build` holds back so the app offers the source as a link until a later pass recovers it. A connection-level failure is retried once (a transient is not drift evidence); an HTTP status, a robots refusal or a parse failure is the source's own answer. A missing SmartBrain_3000 checkout (the csv / feed / xml / text parsers) is now a SKIP for every command, never a FAIL — before this the weekly job would have drifted every non-JSON source within two weeks. Each file keeps its own formatting, so a recheck diff is one line. The weekly job re-checks every file with the app's parsers checked out beside the Library, pushes any changed answers files to recheck/<date> for a PR, and turns red when a file failed or drifted. The live check (answers-check) still exercises drifted files so a recovery can be seen on demand. Five answers files carry today's `checked` from the smoke runs (JSON, CSV and feed sources all pass). Round 19 execution plan, Phase 2a (decision-free part).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Round 19 execution plan, Phase 2a: the part that needs no ruling. The Library gains the drift detection the authoring loop was missing: until now
checkedwas never updated after an answers file was created, and nothing noticed when a source's response changed.What changes
sourcetool answers-recheck [--since DAYS] [ID ...]re-fetches each answered source's live sample (cache bypassed) and runs the same check asanswers-check, then writes the verdict into the file. PASS stampscheckedwith today's date and drops any failure keys. FAIL is remembered inlast_failure. A second failure on a later day marks the filestatus: "drifted";buildholds it back and the app offers the source as a link until a later pass recovers it. Each file keeps its own indent and final newline, so a recheck diff is one line.fetch_sampleborrows the app's csv / feed / xml / text parsers from a sibling SmartBrain_3000 checkout. Without one, every command now skips those sources instead of failing them. Before this fix the weekly job, which has no such checkout, would have drifted all 87 non-JSON sources within two weeks.revalidate, Mondays): re-validates curated records as before, then re-checks every answers file with the app's parsers checked out sparsely beside the Library, pushes any changed files torecheck/<date>for a PR you open, writes the log to the step summary, and turns red when a file failed or drifted. The job gainscontents: writefor that push only.statusandlast_failure, written only by the recheck, validated byload_answers; drifted files are excluded from build and the lints but still seen by the live check. SCHEMA.md documents the rule.Verified
sourcetool checkclean;answers-check --lint0 problems; 255 tests (7 new: state transitions, formatting preserved,--since, held-back files, key validation, parser skip, retry).nws-alerts-point,open-meteo-forecast(JSON),fred-apu0000702111(CSV),npr-news,bbc-world(feeds) all PASS; theircheckeddates in this PR come from those runs.Not in this PR (waits on the plan's rulings)
answers-draft(decision 2: which model drafts),expectsfor the first subcategories (decision 3), the gallery and publish verdict (needs the app's forms from Phase 1a, decision 1).