Skip to content

Correct edit scoring and integrate AuK prompt enhancement - #9

Merged
whk-sjtu merged 3 commits into
mainfrom
codex/issue-8-scoring-missing-target
Oct 26, 2026
Merged

whk-sjtu merged 3 commits into
mainfrom
codex/issue-8-scoring-missing-target

Conversation

@whk-sjtu

@whk-sjtu whk-sjtu commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Requested edits remain in source-qualified scoring denominators when candidate output is missing. Pitch and rate report execution and bounded success relative to the requested change; pause and F0 preservation retain valid source references. Synthetic tests cover missing intervals, partial output, boundary pauses, and compositional aggregation.

AuK uses the official Prompt Enhancer with cached instructions and full-utterance generation. Emotion instructions specify the requested type, with native strength supplied by the candidate compiler. Documentation, reference tables, and radar figures describe and report the current evaluation behavior.

Distribution metadata supplies the software version for API documentation and provenance. Materialization validity follows data, audio, provider, and model contracts; software provenance is informational. Candidate environments use the local package source with locked dependencies.

Validation: 636 tests passed, including materialization provenance compatibility. Compiler and text provenance checks passed. Wheel and source distribution built successfully; installed-wheel metadata and CLI smoke checks passed. Root, six candidate, and five service environment lock checks passed. Three materialization environment lock checks require upstream submodules unavailable in the local checkout. SVGs match the validated result artifacts.

Fixes #8

@whk-sjtu whk-sjtu changed the title Fix missing-output scoring in evaluation protocol v1_1 Score requested edits against source-defined references Oct 15, 2026
Requested edits remain in source-qualified scoring denominators when candidate output is missing. Pitch and rate report execution and bounded success relative to the requested change; pause and F0 preservation retain valid source references. Synthetic tests cover missing intervals, partial output, boundary pauses, and compositional aggregation.

AuK uses the official Prompt Enhancer with cached instructions and full-utterance generation. Emotion instructions specify the requested type, with native strength supplied by the candidate compiler. Documentation, reference tables, and radar figures describe and report the current evaluation behavior.

Fixes #8
@whk-sjtu
whk-sjtu force-pushed the codex/issue-8-scoring-missing-target branch from c153b61 to 2fbf8db Compare October 25, 2026 08:06
@whk-sjtu whk-sjtu changed the title Score requested edits against source-defined references Correct edit scoring and integrate AuK prompt enhancement Oct 25, 2026
@whk-sjtu
whk-sjtu merged commit 9e37899 into main Oct 26, 2026
6 checks passed
@whk-sjtu
whk-sjtu deleted the codex/issue-8-scoring-missing-target branch October 26, 2026 04:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix scoring of missing target measurements and collapsed edit regions

1 participant