[BANADDA]

Text tasks such as Claude Code side jobs can now be scored without new code per task.

rubric_text grader. The rules live in each item's label.rubric:

• format: json requires a JSON object; code fences are stripped first. field names the text being judged.
• equals and set_equals compare fields exactly.
• text_equals compares plain text.
• words and chars bound the length.
• include_any requires at least one key term; exclude bans strings such as refusals.
• case: sentence allows capitals only on the first word, on acronyms, on mixed case words, and on words listed in proper.

An item passes when every check passes. The metric is the share of checks passed. A label with no rubric, or a rubric with no checks, raises GradeError, so a data mistake cannot pass silently.

Generation limits from the arena. The model and bare harness runs used stop=("\n\n", ";") and 256 tokens, which only suit SQL and would cut a summary short. They now read definition.generation.stop and definition.generation.max_tokens, defaulting to the old SQL values, so sql_gen_s1 is unchanged.

First user: the Claude Code asides arena on testnet. The exact session title prompt from Claude Code 2.1.159 maps to a rubric with format json, field title, words [3, 7] and case sentence.

🤖 Generated with Claude Code