Skip to content

answers-draft: model-drafted answers files, promote flow, five promoted sources - #8

Merged
SecureCloudGroup merged 1 commit into
mainfrom
round19/phase2a-draft
Oct 7, 2026
Merged

SecureCloudGroup merged 1 commit into
mainfrom
round19/phase2a-draft

Conversation

@SecureCloudGroup

Copy link
Copy Markdown
Owner

Round 19, Phase 2a (rest): the drafting half of the authoring loop, plus the first expectations and five new answered sources. Follows #7.

What changes

  • sourcetool answers-draft ID ... [--model sonnet|opus|haiku] [--force] — the operator's claude CLI drafts an answers file for a curated source from its live sample; code decides whether it is right. The prompt holds the closed answers spec (rendered from the module constants), the record, its subcategory's keywords and expected components, and a bounded structural sketch of the sample fenced as untrusted data; it never includes a labeled, held-out or sealed ask set. The reply is parsed as JSON and run through the same validators as a hand-written file (answer_problems, check_sample on every row, the kind-served lints), plus two drafting rules the data tests enforce (every value answer declares its window; no utc flag on a path whose values carry a zone). One retry carries the problems back. A passing draft lands in answers/_drafts/<id>.json with draft_meta {tool, model, is_local: false, prompt_sha, date}; a failing one in <id>.rejected.json with the problems. Nothing under _drafts/ is served.
  • sourcetool answers-promote ID ... re-verifies a draft against a fresh live sample and moves it into answers/.
  • sourcetool/claudecli.py mirrors the app's containment for one stateless completion: empty-toolset agent, --setting-sources "", --strict-mcp-config, --no-session-persistence, content over stdin with heading-forgery quoting, private 0700 working directory, no ANTHROPIC_* keys in the environment, process-group kill at 300 s. Publisher-side only; is_local is false in every draft.
  • Expected components (ruling D3, limited to components the app's _lacks already recognises): weather/current [temperature, conditions], water/tides [tide, start_time].
  • Weekly job: the borrowed app parsers import cryptography and numpy; both are now installed (found by running the drafter against the CSV sources).

The first drafting round (18 keyless sources without answers)

Every draft was read by eye against the live sample; the validator proves paths and types, never ordering or window semantics (two passing drafts were wrong by ordering and were corrected by hand). Notes in answers/_drafts/REVIEW.md, skips in answers/_skipped.md.

  • Promoted (5): nhl-club-schedule, openf1-sessions, nvd-cves-keyword, fdc-food-search, sec-companyfacts. Answered sources: 254 → 259.
  • Held (3): ercot-supply-demand (current reading at a moving index of an oldest-first series), treasury-mts (mixed line items), usaspending-awards-agency ("latest populated fiscal year" is not expressible).
  • Dead (1): met-search answers HTTP 410.
  • Rejected for spec or parser gaps (9): numbers like -.19, padded cells and units rows, US dates, split year/month columns, positional pair rows, id-only results, keys with spaces, text products. Listed in REVIEW.md as spec v1.3 candidates; each needs the app's engine to agree first.

Verified

sourcetool check clean; answers-check --lint 0 problems (the keyword baseline shrank 474 → 471: the promoted files speak to three more keywords); 265 tests (10 new for drafting, promotion, the sketch, the prompt's inputs, the CLI event parser and command shape, draft_meta validation); the five promoted files passed a fresh live check at promotion.

…tained claude CLI, promote flow, expects for weather/current and water/tides, five promoted sources

sourcetool answers-draft writes a draft answers file from a live sample with every path and type verified on every row before it is written (draft_meta records tool, model, is_local=false, prompt sha); answers-promote moves a reviewed draft into answers/. Promoted: nhl-club-schedule, openf1-sessions, nvd-cves-keyword, fdc-food-search, sec-companyfacts; three drafts held, nine spec-gap rejections and the review notes under answers/_drafts; weekly job installs the app parsers' dependencies.
@SecureCloudGroup
SecureCloudGroup merged commit 7029f4d into main Oct 7, 2026
3 checks passed
@SecureCloudGroup
SecureCloudGroup deleted the round19/phase2a-draft branch October 7, 2026 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant