Skip to content

data(qp-topics): batch 6 — 847 new qp-topic-v0 label(s) by claude-code/claude-fable-5 (5435 total) - #1960

Merged
modelmirror merged 1 commit into
mainfrom
qp-topics/refresh
Sep 21, 2026
Merged

modelmirror merged 1 commit into
mainfrom
qp-topics/refresh

Conversation

@fedcourtsai-dev

Copy link
Copy Markdown
Contributor

Batch 6 of the qp-topic-v0 labeling frame, by claude-code/claude-fable-5 from the stored questions-presented text alone: 847 new label(s), bringing the artifact to 5435. The artifact accrues one batch a run, so the block below is this batch's measurement and not the file's — the rest of what this batch read is the hand reference set, re-graded every run for that measurement and published from its own adjudicated labels rather than the labeler's. Prior batches' rows are carried forward unchanged.

The command enforces the publication gate, so this PR exists only because the run cleared it. What the numbers mean — agreement with a single v0 reference rater, not accuracy; a grant-enriched frame that certifies the grant stream only; per-label rates suppressed under the support floor — is in docs/qp-topic.md.

qp-topics: batch 6 — 1200 row(s) read by claude-code/claude-fable-5, 847 newly published from its own calls and 0 from the reference set's hand labels; 5435 labeled case(s) in the artifact across 6 batch(es)
  overall:   317/353 (89.8%) vs the v0 reference rater — agreement, not accuracy
  floor:     22.9% — what a constant labeler scores on the same entries
  uncovered: 0 reference entr(ies) the labeler did not cover
    administrative-law-and-benefit-programs 22/29 (75.9%)
    business-and-financial-regulation      19/21 (90.5%)
    civil-procedure                        48/55 (87.3%)
    constitutional-rights                  31/36 (86.1%)
    criminal-law                           80/81 (98.8%)
    election-law                           5/5 (unmeasured in v0)
    employment-and-antidiscrimination      16/18 (88.9%)
    environment-energy-and-property        5/6 (unmeasured in v0)
    firearms                               34/35 (97.1%)
    first-amendment                        12/13 (92.3%)
    habeas-and-postconviction              14/14 (100.0%)
    immigration                            3/5 (unmeasured in v0)
    intellectual-property                  5/5 (unmeasured in v0)
    sovereignty-and-foreign-relations      10/13 (76.9%)
    tax                                    2/3 (unmeasured in v0)
    unclassifiable                         11/14 (78.6%)
  triangle (rows = reference, cols = constitutional-rights, criminal-law, civil-procedure):
    constitutional-rights     31   0   4  other=1 n=36
    criminal-law               0  80   0  other=1 n=81
    civil-procedure            1   0  48  other=6 n=55
  shadow:    47 disagreement(s) on 189 rule firing(s) over 1200 text(s) — the rules are unmeasured off the reference set, so only the movement between runs of one labeler reads
qp-topics: gate passed -> data/qp-topics/qp-topics.json

Run: https://github.com/ModelMirrorAI/fedcourtsai/actions/runs/35650243088

@modelmirror
modelmirror merged commit 460d21c into main Sep 21, 2026
7 checks passed
@modelmirror
modelmirror deleted the qp-topics/refresh branch September 21, 2026 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant