diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 74ca6c1..bc9abc3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -43,6 +43,9 @@ jobs: with: workspaces: src-tauri -> target + - name: Verify Tauri default binary + run: cargo metadata --no-deps --format-version 1 | grep -q '"default_run":"scorebench"' + - name: Check formatting run: cargo fmt --check @@ -52,6 +55,46 @@ jobs: - name: Run tests run: cargo test + scorekit-contract: + name: ScoreKit 0.6 contract + runs-on: ubuntu-latest + steps: + - name: Check out scorebench + uses: actions/checkout@v5 + + - name: Check out pinned ScoreKit + uses: actions/checkout@v5 + with: + repository: talkincode/scorekit + # v0.6.0 release commit; pin the immutable contract, not a moving branch. + ref: 5cea04fb75764480e18759674aae4cb8d917a518 + path: .scorekit-contract + + - name: Install native dependencies + run: | + sudo apt-get update + sudo apt-get install --no-install-recommends -y \ + ffmpeg \ + fluidsynth \ + libwebkit2gtk-4.1-dev \ + libgtk-3-dev \ + libayatana-appindicator3-dev \ + librsvg2-dev + + - name: Install Rust toolchain + uses: dtolnay/rust-toolchain@stable + + - name: Build pinned ScoreKit + run: cargo build --locked --manifest-path .scorekit-contract/Cargo.toml + + - name: Run non-skippable ScoreKit contract + env: + SCOREBENCH_SCOREKIT: ${{ github.workspace }}/.scorekit-contract/target/debug/scorekit + SCOREBENCH_REQUIRE_SCOREKIT_CONTRACT: "1" + run: >- + cargo test --manifest-path src-tauri/Cargo.toml + scorekit_06_contract_tools_query_live_capabilities -- --nocapture + frontend: name: Frontend runs-on: ubuntu-latest diff --git a/.gitignore b/.gitignore index 0d980fb..0f5e07d 100644 --- a/.gitignore +++ b/.gitignore @@ -12,3 +12,6 @@ vite.config.ts.timestamp-* /examples/mood-demo/main.js /examples/armor-material-lab/main.js /examples/voyage-demo/main.js + +# Per-machine workspace path overrides (see scorekit-workspace.json in the scorekit repo) +scorekit-workspace.local.json diff --git a/README.md b/README.md index b51da2f..0d7d785 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,7 @@ Install scorebench from [Releases](https://github.com/talkincode/scorebench/rele - Linux: use the `.deb` or `.AppImage` artifact. - Windows: use the `.msi` or `-setup.exe` artifact. -Install ScoreKit 0.3.x–0.4.x separately (for example with Homebrew `brew install talkincode/tap/scorekit`, or from [ScoreKit Releases](https://github.com/talkincode/scorekit/releases)), then confirm: +Install ScoreKit 0.5.x separately (for example with Homebrew `brew install talkincode/tap/scorekit`, or from [ScoreKit Releases](https://github.com/talkincode/scorekit/releases)), then confirm: ```bash brew trust --tap talkincode/tap # when installing ScoreKit via Homebrew tap @@ -98,6 +98,11 @@ npm install npm run tauri dev # requires Rust toolchain + scorekit on PATH ``` +Working across the whole constellation (scorekit, samples, forge, ScoreData)? +The committed multi-repo map is [`scorekit-workspace.json` in the scorekit +repo](https://github.com/talkincode/scorekit/blob/main/scorekit-workspace.json); +run `python3 scripts/workspace.py doctor` there to check/bootstrap the layout. + ## Documentation The English user guide covers the ScoreKit scene protocol, practical diff --git a/docs-site/src/getting-started.md b/docs-site/src/getting-started.md index 7dcd68c..834702c 100644 --- a/docs-site/src/getting-started.md +++ b/docs-site/src/getting-started.md @@ -4,7 +4,7 @@ This chapter takes the shortest path through the complete loop: install the depe ## 1. Install scorebench and ScoreKit -scorebench needs the ScoreKit CLI at runtime. The desktop app does not bundle the CLI. This version is tested with ScoreKit 0.3.x–0.4.x. +scorebench needs the ScoreKit CLI at runtime. The desktop app does not bundle the CLI. This version is tested with ScoreKit 0.5.x. ### macOS (Homebrew, recommended) diff --git a/docs-site/src/glossary.md b/docs-site/src/glossary.md index 2ab49c9..488ab4d 100644 --- a/docs-site/src/glossary.md +++ b/docs-site/src/glossary.md @@ -31,7 +31,7 @@ | Arrangement | Assigning melody, harmony, bass, rhythm, color, and form to instruments. | | Texture | How simultaneous voices are organized, from sparse solo writing to dense layers. | | Register | Whether material sits in a low, middle, or high pitch range. | -| Section | A named suite cue that can change length, tempo, looping, overall intensity, and muted tracks. | +| Section | A named suite cue that can change length, tempo, looping, overall intensity, and tracks muted by stable `id`. | ## Performance and space @@ -55,7 +55,9 @@ | Renderer | Software that converts MIDI plus a sound source into PCM audio, such as FluidSynth, TiMidity++, or sfizz. | | SoundFont / SF2 | A file that packages samples and mappings for multiple, often GM-compatible, instruments. | | SFZ | A text instrument format that maps external WAV or FLAC samples. | -| Renderer profile | YAML that maps ScoreKit instruments and articulations to local SFZ files. | +| Renderer profile | YAML that maps ScoreKit instruments and articulations to local SFZ files; a leaf bound into an orchestration's palettes. | +| Orchestration profile | YAML routing each scene track's logical `palette` (or the orchestration's `default_palette`) to a renderer profile; only meaningful for `--renderer sfizz`. | +| Palette | A named logical role (for example `default`, `solo`) an orchestration binds to one renderer profile; scene tracks select one via `palette`. | | General MIDI / GM | Standard program and drum-channel conventions used to map scene instruments into compatible SoundFonts. | | Sample | A digital recording used by a sampled instrument. | | Sample rate | Audio samples per second, commonly 44,100 or 48,000 Hz. | diff --git a/docs-site/src/rendering.md b/docs-site/src/rendering.md index 0429069..0d6129f 100644 --- a/docs-site/src/rendering.md +++ b/docs-site/src/rendering.md @@ -10,7 +10,7 @@ MIDI │ ├── FluidSynth + SF2 ├── TiMidity++ + SF2 - └── sfizz_render + SFZ profile + └── sfizz_render + orchestration (palette → renderer profile) + SFZ │ ▼ synth audio ──┐ @@ -30,7 +30,7 @@ ScoreKit launches the tools, checks their results, and writes artifacts atomical | --- | --- | --- | --- | | `fluidsynth` | One GM-compatible SF2 | Default sketches, fast iteration, easy setup | Every instrument comes from one SoundFont; `articulation` has no effect | | `timidity` | One GM-compatible SF2 | An alternate SF2 path or cross-check | Mixing and controller response can differ; `articulation` has no effect | -| `sfizz` | SFZ files plus a renderer profile | Detailed sample libraries and multiple articulations | Requires a profile and a mapping for every scene instrument | +| `sfizz` | SFZ files plus an orchestration profile | Detailed sample libraries and multiple articulations | Requires an orchestration mapping every track's palette; single-instrument renders can use `--sfz` directly | FluidSynth is the default. Switch to sfizz only when the musical structure is stable and you are ready to manage sample libraries, file paths, compatibility, and licensing. @@ -38,16 +38,16 @@ FluidSynth is the default. Switch to sfizz only when the musical structure is st | Control | Meaning | Guidance | | --- | --- | --- | -| Renderer | The MIDI-to-PCM synthesis backend | Start with FluidSynth; sfizz requires a profile | +| Renderer | The MIDI-to-PCM synthesis backend | Start with FluidSynth; sfizz requires an orchestration profile | | Sample rate | 44,100 or 48,000 samples per second | 44.1 kHz is common for music; 48 kHz is common in video and some game pipelines | | Format | `OGG` or `WAV` | OGG is compact; WAV is lossless and better for later production | | Gain | Overall renderer amplitude, from 0 to 2 in the UI | Start at 0.8; reduce clipping here, but do not use gain to fix arrangement balance | | Quality | Vorbis quality from 0 to 10 | Primarily affects OGG size and encoding quality; default 5 | | Stems | Render every track as aligned audio | Enable for adaptive playback or downstream mixing | -| SFZ profile | Maps instruments and articulations to `.sfz` files | Used only by sfizz; missing mappings fail the build | +| Orchestration profile | Routes each track's logical `palette` to a leaf renderer profile mapping instruments and articulations to `.sfz` files | Used only by sfizz; a missing palette or unmapped instrument fails the build | | Texture profile | Maps portable texture source names to local audio files | Renderer-independent; required only when the scene declares `textures` | -The renderer, SFZ profile, and texture profile are stored in the project's `bench.json`, allowing the Agent to check instrument and texture-source compatibility while writing a scene. The other controls are immediate Render-panel choices. +The renderer, orchestration profile, and texture profile are stored in the project's `bench.json`, allowing the Agent to check per-track palette and texture-source compatibility while writing a scene. The other controls are immediate Render-panel choices. ## Output files @@ -79,5 +79,5 @@ A suite with `sections` emits separate audio assets per section. Every stem is s 1. Still changing melody, harmony, or form? Use FluidSynth. 2. Want to compare a second SF2 renderer? Try TiMidity++. -3. Need detailed samples or multiple articulations? Prepare and certify an SFZ profile, then use sfizz. +3. Need detailed samples or multiple articulations? Prepare and certify a renderer profile, bind it into an orchestration palette, then use sfizz. 4. Need commercial plugins, a DAW effect chain, or mastering? Export WAV or stems and continue in an external production workflow. diff --git a/docs-site/src/scene-protocol.md b/docs-site/src/scene-protocol.md index 396cdff..9961d45 100644 --- a/docs-site/src/scene-protocol.md +++ b/docs-site/src/scene-protocol.md @@ -37,17 +37,21 @@ motifs: - { degree: 0, beats: 4 } tracks: - - instrument: flute + - id: lead + instrument: flute pattern: melody motif: forest_call intensity: 0.45 - - instrument: slow_strings + - id: harmony + instrument: slow_strings pattern: sustain intensity: 0.35 - - instrument: harp + - id: motion + instrument: harp pattern: arpeggio intensity: 0.3 - - instrument: bass + - id: foundation + instrument: bass pattern: bass intensity: 0.3 ``` @@ -74,9 +78,9 @@ tracks: ScoreKit currently builds diatonic triads from the scene scale. Roman-numeral case is conventional: `VI` and `vi` select the same scale degree in the current protocol. This field is not a complete classical-harmony notation system. -## Tracks and the five patterns +## Tracks and the six patterns -Every track selects an `instrument` and a `pattern`: +Every track selects an `instrument` and a `pattern`, and carries a stable scene-local `id`: | Pattern | Generated material | Typical role | | --- | --- | --- | @@ -85,9 +89,12 @@ Every track selects an `instrument` and a `pattern`: | `arpeggio` | Eighth notes in root–third–fifth–third order | Motion, piano or harp figures | | `bass` | Low roots derived from the current chord | Low-frequency foundation | | `drums` | A fixed kick, snare, and hi-hat groove | Basic pulse; must use `instrument: drums` | +| `tabla` | A deterministic tabla groove | World-percussion pulse; must use `instrument: tabla` and still obey the single-percussion-track rule | Tracks can also define: +- `id`: a stable identifier matching `[a-z][a-z0-9_-]{0,63}`. Required; referenced by section `mute` lists, stems, and MIDI's `--solo` selector — never a positional index, so reordering tracks never breaks a reference. +- `palette`: an optional logical orchestration palette name. Absent uses the active orchestration's `default_palette`; irrelevant when not building with `--renderer sfizz --orchestration ...`. Routing metadata only — it never changes compiled MIDI. - `intensity`: velocity scaling from 0.0 to 1.0. - `articulation`: `sustain`, `staccato`, `spiccato`, `pizzicato`, `tremolo`, or `mute`. It selects SFZ samples only; SF2 backends ignore it. - `pan`: 0.0 hard left, 0.5 center, 1.0 hard right, compiled to MIDI CC10. @@ -108,7 +115,7 @@ A melody repeats or truncates its motif to fill the scene or section exactly. Sp ## Sound textures -ScoreKit 0.3 adds deterministic non-instrument layers: +ScoreKit provides deterministic non-instrument layers: ```yaml textures: @@ -127,12 +134,22 @@ textures: ```yaml name: forest-recordings root: recordings +schema_version: 1 sources: - river: river.flac - birds: birds.wav + river: + path: river.flac + description: Steady close river + category: organic + tags: [water, flowing] + playback: + modes: [loop] + default_mode: loop + use_cases: [forest] + provenance: + library: field-recordings@1.0.0 ``` -The Agent receives the active profile's source keys and the observation panel reports missing mappings before build. ScoreKit still performs the authoritative validation and mixing. Enabling stems produces aligned texture stems alongside instrument stems. +ScoreKit 0.6 also accepts the old path-only binding for build compatibility, but discovery and certification require the structured form. The Agent queries the active profile with exact `inspect_textures` filters before choosing a source; `no_match` means it must change the plan, not invent a key. The observation panel reports missing mappings and a `loop`/`one_shot` mode not declared by the source before build. ScoreKit still performs authoritative validation and mixing. Enabling stems produces aligned texture stems alongside instrument stems. ## Performance @@ -158,17 +175,17 @@ performance: ```yaml sections: - - { name: intro, bars: 4, loop: false, mute: [2, 3], intensity: 0.7 } - - { name: explore, bars: 8, loop: true, mute: [3], intensity: 0.9 } + - { name: intro, bars: 4, loop: false, mute: [motion, foundation], intensity: 0.7 } + - { name: explore, bars: 8, loop: true, mute: [foundation], intensity: 0.9 } - { name: combat, bars: 8, loop: true, intensity: 1.25, tempo: 108 } - - { name: victory, bars: 4, loop: false, mute: [3], intensity: 1.1 } + - { name: victory, bars: 4, loop: false, mute: [foundation], intensity: 1.1 } ``` -A section can change `bars`, `tempo`, `loop`, and overall `intensity`, or silence tracks through the **zero-based** `mute` indexes. Sections inherit the top-level key, harmony, motifs, tracks, and performance. The current protocol cannot replace harmony or motif contents per section. +A section can change `bars`, `tempo`, `loop`, and overall `intensity`, or silence tracks through the **stable track `id`s** listed in `mute`. Sections inherit the top-level key, harmony, motifs, tracks, and performance. The current protocol cannot replace harmony or motif contents per section. ## What does not belong in the scene protocol -- SoundFont, SFZ, renderer, and recording paths. They belong in build parameters, renderer profiles, or texture profiles; scene textures use portable source keys. +- SoundFont, SFZ, renderer, orchestration, and recording paths. They belong in build parameters, orchestration/renderer profiles, or texture profiles; scene textures use portable source keys. - Arbitrary `mood`, `danger`, or `avoid` fields without compile semantics. Keep them in the conversation or `story`. - Plugin chains, mastering, equalization, or post-processing instructions. - Arbitrary MIDI events, automation curves, or free-form per-note editing outside the schema. diff --git a/docs-site/src/sound-sources.md b/docs-site/src/sound-sources.md index f127585..9524930 100644 --- a/docs-site/src/sound-sources.md +++ b/docs-site/src/sound-sources.md @@ -36,13 +36,15 @@ Before adopting a custom SF2, remember: - `articulation` does not switch an SF2 to pizzicato, spiccato, or another sample set. Only SFZ profiles use that field. - A license may permit music made with a library while forbidding redistribution of the original samples or instrument file. +ScoreKit 0.6's scene vocabulary includes world identities such as `erhu`, `pipa`, `guzheng`, `dizi`, `tabla`, `oud`, `ney`, and `duduk`. They validate as musical identities but do not have an exact General MIDI program, so an SF2 fallback is intentionally rejected. `shakuhachi`, `shamisen`, and `sitar` do have exact GM programs. Run `scorekit inspect-instruments scene.yaml --json` before promising any world identity; use an orchestration with an exact source when the default SF2 cannot provide one. + The current scorebench Render panel does not expose a custom-SoundFont picker, although the Agent/backend build interface supports a `soundfont` parameter. For a fixed project source, configure the ScoreKit sound directory or make the build workflow pass an explicit path rather than copying a large library into the repository. ## SFZ files and samples SFZ is a text format that describes how samples are mapped into an instrument. A `.sfz` commonly references WAV or FLAC files beside it or elsewhere in the library. Copying the `.sfz` without its referenced samples cannot produce sound. -ScoreKit uses a renderer profile to map protocol instruments onto local SFZ files: +ScoreKit uses a renderer profile to map protocol instruments onto local SFZ files, and a multi-profile **orchestration** to route each scene track's logical `palette` to one of these renderer profiles: ```yaml name: open-orchestra @@ -59,7 +61,7 @@ instruments: Every instrument needs a `sustain` mapping. When a dedicated articulation is absent, ScoreKit falls back to that instrument's `sustain`. This keeps machine-specific paths out of portable scene YAML, but teams still need an installation convention for the profile and its sample root. -Certify a profile before use: +Certify a renderer profile before use: ```bash scorekit profile check profile.yaml @@ -68,6 +70,24 @@ scorekit --json profile check profile.yaml The check covers paths, render probes, silence, warnings, and repeat-render behavior. One successful patch does not prove that every patch in the same library is compatible with sfizz. +Bind one or more certified profiles into an orchestration, then validate the whole routing (palette bindings, leaf profiles, and every referenced SFZ file): + +```yaml +schema_version: 1 +name: hybrid-cinematic +default_palette: default +palettes: + default: { profile: ../renderers/scoredata-open.yaml } + solo: { profile: ../renderers/scoredata-chamber.yaml } +``` + +```bash +scorekit orchestration check hybrid-cinematic.yaml +scorekit build scene.yaml --renderer sfizz --orchestration hybrid-cinematic.yaml -o out/scene.ogg +``` + +A scene track without a `palette` uses `default_palette`; a track with `palette: solo` routes through the `scoredata-chamber` profile instead, without any DSL field naming a sound source. + ## Open sources worth evaluating ScoreKit's example profiles and research have used or evaluated sources such as: diff --git a/docs-site/src/troubleshooting.md b/docs-site/src/troubleshooting.md index 4cf7457..33f207c 100644 --- a/docs-site/src/troubleshooting.md +++ b/docs-site/src/troubleshooting.md @@ -91,28 +91,29 @@ Ask the Agent to repair the exact `field` and `message` from the structured erro Check in this order: -1. A profile is selected in the Render panel. -2. The profile `root` and relative `.sfz` paths exist. -3. Every instrument in the scene has a profile mapping. -4. Every mapped instrument has at least `sustain`. -5. `sfizz_render` is on `PATH`. -6. `scorekit profile check profile.yaml` succeeds. -7. Every WAV or FLAC referenced by the SFZ is present. +1. An orchestration profile is selected in the Render panel. +2. Every track's declared (or default) `palette` exists in the orchestration. +3. Each palette's leaf renderer profile is readable, and its `root` and relative `.sfz` paths exist. +4. Every instrument in the scene has a mapping in its track's resolved palette. +5. Every mapped instrument has at least `sustain`. +6. `sfizz_render` is on `PATH`. +7. `scorekit orchestration check orchestration.yaml` and `scorekit profile check profile.yaml` succeed. +8. Every WAV or FLAC referenced by the SFZ is present. -A missing dedicated articulation falls back to sustain. A completely unmapped instrument fails the build. +A missing dedicated articulation falls back to sustain. A completely unmapped instrument, or a track referencing an undefined `palette`, fails the build; the observation panel and `write_scene`/`validate_scene` tool output surface exactly which track and which problem. ## Articulation does not change the sound -This is expected with FluidSynth or TiMidity++: `articulation` does not change MIDI and cannot switch SF2 programs. Only an sfizz renderer profile can map articulations to different `.sfz` files. +This is expected with FluidSynth or TiMidity++: `articulation` does not change MIDI and cannot switch SF2 programs. Only an sfizz renderer profile (bound through an orchestration palette) can map articulations to different `.sfz` files. -With sfizz, verify that the profile has a dedicated mapping for the requested articulation. Otherwise ScoreKit deliberately falls back to sustain. +With sfizz, verify that the resolved palette's renderer profile has a dedicated mapping for the requested articulation. Otherwise ScoreKit deliberately falls back to sustain. ## The build succeeds but an instrument is wrong or silent - For SF2, confirm that the file is valid and has complete GM mappings. - For SFZ, verify that the patch's playable range covers the scene notes. - Check gain, track intensity, and section intensity for near-zero values. -- Check whether a section's zero-based `mute` list silences the track. +- Check whether a section's `mute` list silences the track by its stable `id`. - Inspect the structured ScoreKit error and matching `meta.json`, not only the process exit code. ## The loop seam sounds abrupt @@ -151,7 +152,7 @@ Include: - Operating system and architecture. - Redacted output from `scorekit --json doctor`. - A minimal scene YAML that reproduces the issue. -- Renderer, sample rate, format, and profile name. +- Renderer, sample rate, format, and orchestration/profile name. - The complete structured error. - Whether default FluidSynth plus MuseScore General also reproduces the issue. diff --git a/docs-site/src/working-with-agent.md b/docs-site/src/working-with-agent.md index 37ec068..cedbbd3 100644 --- a/docs-site/src/working-with-agent.md +++ b/docs-site/src/working-with-agent.md @@ -10,7 +10,7 @@ Mood: What should the listener feel, and what should be avoided? Length and form: How many bars, and are intro/explore/combat sections needed? Material: Tempo, key, meter, motif, or a reference color. Orchestration: Who carries melody, harmony, bass, rhythm, and texture? -Constraints: Do you need stems, a seamless loop, or a particular SFZ profile? +Constraints: Do you need stems, a seamless loop, or a particular orchestration profile? Acceptance: What would make the result complete? ``` @@ -22,7 +22,7 @@ You do not need to know every theory term. A phrase such as “a distant memory, 2. **Generate and validate the scene.** Passing `scorekit validate` proves protocol validity, not musical quality. 3. **Listen with the default sound source.** Use FluidSynth to judge melody, harmony, density, and form quickly. 4. **Change one class of problem at a time.** Fix phrasing, then orchestration, then sound sources. The semantic diff remains meaningful. -5. **Move to the target sound source last.** An SFZ profile can change envelopes, balance, articulation, and perceived space, so listen again after switching. +5. **Move to the target sound source last.** An orchestration profile can change envelopes, balance, articulation, and perceived space, so listen again after switching. 6. **Use Review for alternatives.** Review is text analysis over an evidence pack. It does not hear the audio or edit the scene directly. ## Make revision requests executable @@ -35,17 +35,17 @@ You do not need to know every theory term. A phrase such as “a distant memory, | Make the ending natural | For a loop, make the final harmony return to the opening; for a one-shot, leave a clear decay | | Make it sound human | Add small seeded timing and velocity variation, then verify legato and the target articulation | -## StylePack, grammar, and renderer profile are different +## StylePack, grammar, and orchestration profile are different -- A **StylePack** is a scorebench creative preference package. It guides the Agent toward particular harmony, orchestration, form, and review criteria. +- A **StylePack** is a scorebench creative preference package. It guides the Agent toward particular harmony, arrangement, form, and review criteria. - A **grammar profile** is a set of measurable ScoreKit aesthetic checks, such as tempo limits, voice count, and melody rest ratio. -- A **renderer profile** maps ScoreKit instruments and articulations to local SFZ files for sfizz. +- An **orchestration profile** routes each scene track's logical `palette` (or the orchestration's `default_palette`) to a leaf renderer profile mapping ScoreKit instruments and articulations to local SFZ files for sfizz. - A **texture profile** maps portable ambience and sound-effect source names to local audio files, independently of the renderer. -StylePack influences choices, grammar checks the compiled music, a renderer profile controls instrumental timbre, and a texture profile binds scheduled recordings. None of them replaces the others. +StylePack influences choices, grammar checks the compiled music, an orchestration profile controls instrumental timbre per track, and a texture profile binds scheduled recordings. None of them replaces the others. ## Project files and recovery -Scenes, the project manifest, sessions, and Agent memory are plain files. When the Agent writes a scene, scorebench records semantic history and validates immediately. Invalid YAML may remain on disk with a visible error so the Agent can repair it. Put the project under Git if you want review and rollback. +Scenes, the project manifest, sessions, and Agent memory are plain files. When the Agent writes a scene, scorebench records semantic history and validates immediately. Invalid YAML may remain on disk with a visible error so the Agent can repair it, but independent validation, build, instrument-resolution, and grammar gates prevent that run from finishing normally until every failed check is retried successfully. Streamed model text remains visibly marked as a provisional draft until the response and any tools finish. Accepted text stays transactional across the run and is finalized only after the completed history is durable on disk; blocked, incomplete, cancelled, exhausted, failed, or non-durable claims are removed instead of being presented as completed chat history. Project switching is disabled during a live run, and late events are tied to the project and session that started it. Put the project under Git if you want review and rollback. -The selected renderer, SFZ profile, and texture profile are stored in `bench.json`. Sample rate, gain, quality, format, and stems are immediate Render-panel choices. Never put secrets in `bench.json`. +The selected renderer, orchestration profile, and texture profile are stored in `bench.json`. Sample rate, gain, quality, format, and stems are immediate Render-panel choices. Never put secrets in `bench.json`. diff --git a/docs/roadmap.md b/docs/roadmap.md index 9a8b605..aaee77a 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -74,15 +74,22 @@ Preset styles are structured **StylePacks**, not prompt strings: YAML documents The pack is injected into the system prompt with a mandatory conflict-detection protocol: the agent must check each request against the active style, surface conflicts and propose a reconciliation (or ask) instead of silently implementing, with explicit user instructions winning after the conflict is acknowledged. `review.criteria` feed the review panel's evidence pack so a pack also carries its own acceptance standards. This replaced the free-text "persona" setting, which conflicted with per-request intent; the legacy settings field still parses but is no longer injected — parsing is scheduled for removal in 0.5.0 (field, validation, and round-trip tests drop together). -### M8 — Capability acceptance (status: planned) +### M8 — Capability acceptance (status: in progress — offline schema runner shipped) The agent's translation capability (intent → scene edits) is a black box that cannot be opened, but its boundary can be accepted. Two layers; only the second is new: -**Loop contract (deterministic — largely shipped).** Every write passes `scorekit validate`; every edit yields a semantic diff; the mood spectrum maps rendered audio into V-A-T emotion coordinates. These prove the loop is intact — not that the translation is good. +**Loop contract (deterministic — shipped for Agent writes).** Every Agent write runs `scorekit validate`; every edit attempts a semantic diff; active orchestration and texture source/mode compatibility are checked; and a request-local terminal gate withholds normal completion until every touched path is ready. The mood spectrum maps rendered audio into V-A-T emotion coordinates. These prove the loop is intact — not that the translation is good. -**Capability evals (statistical — the missing layer).** A fixed set of intent cases ("lonelier", "a calm 30s menu loop"), each carrying machine-checkable **directional assertions**: schema-level (tempo decreased, mode moved to minor, arrangement thinned — plain YAML checks) and signal-level (V-A-T coordinates move in the intended direction — the mood features are pure functions and run offline). Each model/prompt change runs the set N times and reports pass rates; the tracked quantity is *regression*, not perfection — pass rates below 100% are expected. The harness is a separate opt-in runner outside `cargo test`, preserving the no-live-LLM-in-tests rule. Directional assertions are the deliberate design choice: absolute musical-quality scoring is rejected — it would re-import the jury/consensus machinery already rejected in M6. +**Capability evals (statistical — schema runner shipped, generation and signal layer open).** The opt-in `capability_eval` binary reads a fixed intent case, one baseline Scene, and N saved candidate Scenes, then reports per-sample assertions and pass rate. The first case covers tempo direction, mode, arrangement thinning by track count, melody rest ratio, loop preservation, and motif-contour preservation. It never calls an LLM, ScoreKit, or an audio renderer and can enforce a regression threshold through its exit code: -**Intent contract (in-product counterpart).** Before acting, the agent declares falsifiable expectations ("tempo to 68, D minor, thinner pads"); after acting, the existing evidence surfaces (validate status, semantic diff, mood delta) confirm or refute the declaration, closing each loop iteration with user-visible acceptance. This turns black-box capability into a per-action falsifiable claim using surfaces that already exist — the cost is a prompt protocol plus aligning three panels to the declaration, not new infrastructure. +```bash +cargo run --manifest-path src-tauri/Cargo.toml --bin capability_eval -- \ + src-tauri/tests/fixtures/capability/lonelier.case.yaml --min-pass-rate 0.5 --pretty +``` + +Model/prompt orchestration still needs to generate and save N samples outside `cargo test`; signal-level V-A-T direction assertions remain open. The tracked quantity is *regression*, not perfection — pass rates below 100% are expected. Directional assertions are deliberate: absolute musical-quality scoring would re-import the jury/consensus machinery rejected in M6. + +**Intent contract (in-product counterpart — prompt protocol shipped).** The versioned Arrangement Canon requires the Agent to declare falsifiable expectations before writing and compare validation/diff/build evidence afterward. Persisting that declaration as typed UI state and aligning it with the diff and mood panels remain open. This turns black-box capability into a per-action falsifiable claim without treating model self-report as proof. This milestone is the second half of the project's name: scorebench as the *bench* on which the agent's capability is measured. @@ -109,7 +116,7 @@ Boundaries: 3. **Open on the input side, closed on the output side.** MCP brings in the upstream of intent — references, lyrics, game-level metadata, engine integration. Scene writes stay on the built-in `write_scene` path (validate → diff → history is not bypassable by routing), and the only sound outlet remains scorekit. 4. **Trust model.** Servers are user-configured, user-trusted subprocesses (global or per-project config); secrets follow the keychain/untracked discipline; every tool call is visible and auditable in the chat. 5. **Test rule unchanged.** No live MCP servers in tests; dispatch logic is tested against recorded fixtures, same as the LLM rule. -6. **Context budget.** Discovered tool schemas count toward the budget and servers can be toggled; the "stable eight-tool schema" acceptance row evolves into "built-in tools + discovered tools". +6. **Context budget.** Discovered tool schemas count toward the budget and servers can be toggled; the stable built-in tool schema evolves into "built-in tools + discovered tools". Positioning tie-in: this opens the *vocabulary upstream* of the emotion expresser to the user (game context and text material flow in to feed the translator) while the compiler downstream keeps its single outlet — the two-face structure is preserved. @@ -127,18 +134,22 @@ Rules (MUST): 4. Every state-mutating operation (project files, memory, settings) verifies recovery after at least one failure: no corrupted partial state. 5. New tier-1 features land together with their tests and an update to this matrix. +**scorekit 0.5 orchestration migration (2026-07, breaking):** scorekit replaced the single-profile `--profile` sfizz build with multi-profile `--orchestration` routing (`scorekit build --renderer sfizz --orchestration path`, `scorekit orchestration check path`); scene tracks now require a stable `id: [a-z][a-z0-9_-]{0,63}` and accept an optional logical `palette`, and section `mute` lists (plus MIDI's `--solo`) select tracks by that ID instead of a positional index. scorebench followed with no compatibility shim: `bench.json`'s `RenderConfig.profile` became `RenderConfig.orchestration`, the single-profile `ProfileCompat` model became a per-track `OrchestrationCompat` (missing palette, unreadable leaf profile, and unmapped instrument each surface per track_id, never silently), `TrackDisplay`/`SectionDisplay` gained `id`/`palette` and switched `mute` to string IDs, and every scorekit subprocess call and agent tool schema moved from `--profile` to `--orchestration`. The tested scorekit range moved from `>=0.3.0, <0.5.0` to `>=0.5.0, <0.6.0` since no earlier release understands the new contract. The recorded contract fixtures (`doctor.json`, `forest.meta.json`, `error_resolution.json`) were re-taken from a real 0.5.0 binary, and the machine error surface now carries scorekit's `report` verbatim so a routing failure names the track, its palette, its leaf renderer profile, and the substitute the fallback policy rejected instead of only counting unresolved instruments. + +**scorekit 0.6 arrangement-capability migration (2026-07, breaking floor):** the tested range is now `>=0.6.0, <0.7.0`. The floor moves because the composing Agent exposes v0.6-only `inspect-instruments`, `texture inspect`, and `texture check` contracts rather than maintaining a version-dependent tool table. Scorebench reads both legacy string and v0.6 structured texture bindings; its range-bound local preflight rejects malformed structured metadata and unsupported `loop`/`one_shot` scheduling, while discovery/certification remain ScoreKit commands. The built-in Arrangement Canon is a versioned, read-only doctrine separate from StylePacks: it imports the ScoreKit skill's intent contract, palette-inertia check, motif/voice/rest/curve/loop craft constraints, exact-only world-instrument rules, source-honesty rule, and ordered delivery gates. The live schema and machine results remain authoritative. A per-run terminal gate now prevents the Agent from normally finishing while any scene it touched is invalid or incompatible; readiness, build, instrument-resolution, and grammar failures are independent blockers over canonical project-relative identities. One-shot build overrides cannot clear incompatibility in the active project configuration. Text continues to stream as a visibly provisional draft. It is accepted per turn only after tool results prove the gate clear, remains transactional across the run, and is finalized only after history persistence; blocked, incomplete, cancelled, max-turn, failed, or non-durable claims are discarded or rolled back. Frontend events are additionally bound to one run/root/session, and the project cannot switch while that run is active. + | Feature | Tier | Happy path | Failure path | | --- | --- | --- | --- | | scorekit binary discovery | 1 | integration tests run via PATH-located binary | `locate_missing_everywhere_is_typed_error`, `locate_env_override_must_be_executable`, `locate_settings_pin_wins_over_path_and_well_known`, `locate_settings_pin_must_be_executable` (`scorekit.rs`) | -| scorekit error contract (`--json` stderr) | 1 | `doctor_fixture_shape_holds`, `meta_fixture_shape_holds` (recorded fixtures) | `parses_recorded_io_error`, `falls_back_on_non_json_stderr` | -| Build param → CLI arg mapping | 1 | `build_params_render_full_arg_set` covers renderer and `--texture-profile` | n/a (pure function, no state) | +| scorekit error contract (`--json` stderr) | 1 | `doctor_fixture_shape_holds`, `meta_fixture_shape_holds` (0.6.0 recordings: per-track `id`/`palette`/`articulation`, `general_midi_exact` resolution keyed by `track_id`, version inside the tested range) | `parses_recorded_io_error`, `falls_back_on_non_json_stderr`, `keeps_resolution_report_from_orchestration_failure`, `texture_check_failure_keeps_the_certification_report` | +| Build param → CLI arg mapping | 1 | `build_params_render_full_arg_set` covers renderer, `--orchestration`, and `--texture-profile` | n/a (pure function, no state) | | Project directory scan | 1 | `scan_finds_scenes_and_assets` | `scan_rejects_non_directory` | | Asset read containment (webview → disk) | 1 | `resolve_inside` accepts in-root paths | `resolve_inside_blocks_escape` rejects traversal | | Responses SSE transport | 1 | recorded text/tool/multi-tool fixtures; arbitrary chunk-boundary equivalence | 401/429 metadata, failed event, mid-stream disconnect, dead endpoint, cancellation | -| ReACT loop + tool dispatch | 1 | scripted transport writes a scene and completes; stable eight-tool schema | offline transport, unknown/malformed tools, max-turn guard, scorekit tool errors | +| ReACT loop + tool dispatch | 1 | scripted transport repairs a blocked scene and only then completes; stable eleven-tool schema | offline transport, unknown/malformed tools, max-turn guard, invalid-scene completion withheld, scorekit tool errors | | Atomic scene write + semantic history | 1 | real-scorekit diff integration test | rename failure preserves original; history failure warns without blocking edit | | Settings + API key | 1 | settings/keychain round trip | corrupt-file backup, keychain opt-in fallback, atomic-write kill point, invalid hue | -| Scene observation + watcher | 1 | scorekit scene fixtures; external-change GUI smoke | malformed YAML and watcher-storm coalescing | +| Scene observation + watcher | 1 | scorekit scene fixtures; the preview table shows each track's stable `id` and resolved `palette` alongside its instrument, and section segments list the track IDs they mute; external-change GUI smoke | malformed YAML and watcher-storm coalescing | | Project memory + compaction | 1 | repeated three-cycle compaction keeps recent turns and coherent memory | corrupt line recovery; every two-phase kill point restores a loadable generation | | Player + spectrum (WebAudio) | 2 | required GUI smoke: OGG/WAV, seek/pause/loop, live style switching incl. Three.js scenes, auto style, fullscreen visualizer, and one view/canvas surviving embedded ↔ fullscreen; automated: two-entry LRU and 120-frame timing windows | decode errors surface; import/create/resize/render/context-loss falls back to Bars without stopping playback; LRU eviction disposes before replacement creation and remounts the style canvas | | Visualizer video export (webview MediaRecorder) | 2 | mime pick + file naming + 720p/1080p/1440p/4K output sizing unit tests (`recording.test.ts`); watermark wrap/signature unit tests (`titleCard.test.ts`); filter/sink handoff unit tests (`lib.rs`); GUI smoke: fullscreen REC restarts the piece with the title card + logo/repo watermark burned into every frame, stops manually or at natural end, saves H.264/AAC MP4 via native dialog | unsupported recorder yields an inline message; encoder warm-up timeout, cancelled dialog (discard), and missing-destination write are typed errors surfaced in the overlay; style and resolution switching disabled during a take (canvas remount or encoder resize would kill the stream) | @@ -153,11 +164,13 @@ Rules (MUST): | Locale + interface settings | 2 | persist/load round trip incl. `locale`; unknown locale falls back to `en` before write | legacy `personal_instructions` field still parses (persona retired in favor of style packs; parsing scheduled for removal in 0.5.0) | | Spectrum dynamics helpers | 2 | envelope/smoother/impact unit tests (`dynamics.test.ts`) | idle spectrum stays below quiet ceiling; impact decays without audio | | Mood spectrum (emotion-compiler acceptance view) | 2 | perceptual features distinguish major/minor/cluster/fifths chords (`features.test.ts`); V-A-T emotion matrix — sad/happy valence split, tense material raises tension, build-up detected on crescendo but not steady groove, five world archetypes each win on matching material, intent prior nudges valence, dominant switch respects dwell (`mood.test.ts`); Voyage atmosphere mapping separates worlds/weather and all mood-aware Three.js styles expose the shared HUD (`voyage.test.ts`, `index.test.ts`) | empty/zero frames stay neutral without NaN; missing `sampleRate`/`intentMode` options degrade to defaults; shared HUD is read-only and toggleable (`moodHud`) | +| Offline Agent capability regression | 1 | `lonelier.case.yaml` evaluates one passing and one failing saved sample across six directional/preservation assertions and reports `pass_rate: 0.5`; the opt-in CLI exits 0 at threshold 0.5 | malformed baseline/sample is reported without discarding other samples; duplicate/empty case fields are rejected; threshold 0.75 exits 1; runner makes no live LLM, ScoreKit, or audio call | | Review panel (multi-perspective critique) | 1 | recorded report fixture streams through scripted transport and parses into a typed report; evidence pack assembled from scene/validation/meta/memory/intents (`review.rs`) | non-JSON reply, empty-perspective report, unknown perspective id, transport failure, and missing terminal event are typed errors; evidence gathering rejects out-of-root scenes and tolerates unbuilt projects | -| Toolchain validation gate (write → validate → profile check) | 1 | `write_scene` auto-validates via real scorekit and reports inline; scene×renderer/texture-profile compatibility detected from `bench.json` (`tools.rs`, `manifest.rs`); render config injected into system prompt with mapped instrument and texture source keys | invalid scene persists but is reported `invalid` with the machine error; missing scorekit degrades to `unavailable`; unreadable profiles and unmapped instruments/sources surface as warnings, never block the write | -| bench.json render config persistence | 1 | renderer, SFZ profile, and independent texture profile round trip while preserving unknown fields; GUI selection is inherited by `build_scene` when params are omitted | missing file loads defaults silently; corrupt JSON degrades to defaults with a warning, never an error | +| Toolchain validation gate (write → validate → compatibility → terminal gate) | 1 | `write_scene` always validates via real scorekit; scripted tests prove pure text remains multi-delta streaming, incomplete output fails, failed scene checks open independently scoped gates, matching retries clear them, max-turn/blocked runs roll back, persisted history finalizes accepted text, persistence failure rolls it back, explicit build overrides cannot mask active-config incompatibility, `./` and symlink aliases share an identity, and distinct Unix backslash filenames do not collide; per-track orchestration and texture source/mode compatibility come from `bench.json` | invalid scene persists for repair but normal completion is withheld; text accompanying a failed/blocked tool turn is visibly provisional then discarded, and later run/cancellation/max-turn/persistence failure rolls back earlier transactional text; stale events cannot cross run/root/session boundaries and project switching is disabled while busy; validation cannot be disabled; missing scorekit is `unavailable` and blocked; malformed texture metadata, missing palettes, unreadable profiles, unmapped sources, and unsupported texture modes remain visible | +| bench.json render config persistence | 1 | renderer, orchestration profile, and independent texture profile round trip while preserving unknown fields; GUI selection is inherited by `build_scene` when params are omitted | missing file loads defaults silently; corrupt JSON degrades to defaults with a warning, never an error | | Scene story display (scorekit ≥0.2 `story` field) | 2 | `inspect_scene` surfaces optional top-level `story` prose in the observation panel (`observation.rs`) | absent or non-string story renders nothing; field stays informational, never parsed | -| Sound texture composition (scorekit ≥0.3) | 1 | scene observation extracts loop/one-shot texture layers; `texture_sources_are_checked_against_independent_profile`; GUI and Agent builds pass `--texture-profile` and optional stems include texture tracks | textured scenes without a profile, unreadable profiles, and missing portable source keys are diagnosed before build; scorekit remains the final build authority | +| Sound texture composition (scorekit ≥0.6 discovery contract) | 1 | legacy and structured bindings parse; scene observation extracts loop/one-shot layers; `scorekit_06_contract_tools_query_live_capabilities` exercises exact match, truthful no-match, and certification against the pinned real CLI | missing profiles/keys, unsupported source modes, exact no-match, and structured `texture_check` failures remain machine-readable; ScoreKit stays the sole audio decoder/mixer | +| Arrangement Canon + live capability | 1 | typed embedded Canon tests cover intent/palette/craft/world/texture/gate rules; system prompt keeps Canon, live ScoreKit capability, and active StylePack as separate sections; real v0.6 `inspect-instruments` proves standard GM exact resolution | live schema overrides stale doctrine; world instruments without an exact source return ScoreKit `resolution` even under flexible fallback; texture names are never inferred from prompt text | | Creative assist panel (prompt tags) | 2 | tag click appends its bilingual description to the chat input; selections light recommended pairings across categories (`assist.ts` unit tests: unique ids, valid recommends graph, bilingual coverage) | panel writes prompt text only — never scene YAML; empty selection recommends nothing | | Style pack library (structured presets) | 1 | built-in packs parse and validate; user pack save/list/find round trip with atomic write; rename via `previous_id` removes the old file (`styles.rs`) | invalid id/name/YAML rejected before write; built-in ids are read-only (no save/delete); corrupt user file skipped with a warning, never fatal | | Active style injection + conflict detection | 1 | `bench.json` `style.id` round trip (`manifest.rs`); active pack YAML injected into the system prompt with the mandatory conflict-detection protocol; `review.criteria` embedded in review evidence (`agent.rs`, `review.rs`) | dangling style reference degrades to no style with a warning event; chat and review still run | diff --git a/docs/verification.md b/docs/verification.md index d45db46..6bf8568 100644 --- a/docs/verification.md +++ b/docs/verification.md @@ -1,6 +1,6 @@ # Verification record -Last updated: 2026-07-24 (Asia/Shanghai) +Last updated: 2026-07-27 (Asia/Shanghai) This record is evidence for the M0–M5 issue set. It distinguishes automated proof, native-app smoke, and release operations that require external credentials. @@ -86,3 +86,33 @@ This pass closes the 2026-07-24 project-audit findings: version drift across man - `esbuild` is a declared devDependency (`^0.25.12`); the `demo:*` scripts no longer depend on a global install (`npx --no-install esbuild --version` resolves locally). - Roadmap commitments made falsifiable: the legacy `personal_instructions` field's parsing is scheduled for removal in 0.5.0 (M7 + acceptance matrix), and shared weather-helper extraction is recorded as a hard prerequisite for the next M9 imagery module. - Gates after the change: `cargo fmt --check` clean; `cargo clippy --all-targets --all-features -- -D warnings` clean; `cargo test` passed 121 unit tests + the `tauri_boundary` integration test; `npm test` passed 12 Vitest files (141 tests); `npm run check` reported 0 errors and 0 warnings. + +## scorekit 0.5 orchestration evidence (2026-07-25) + +The `--orchestration` migration landed against scorekit 0.5.0 but kept 0.4.0 recordings and no build-time evidence. This pass closes that gap and was run against a real `scorekit 0.5.0` (`~/.local/bin/scorekit`) with the local `/Volumes/WT9/ScoreData` sound library. + +- Recorded contract fixtures re-taken from the 0.5.0 binary: `doctor.json` (`scorekit doctor --json`), `forest.meta.json` (a real `scorekit build` of `tests/fixtures/scenes/forest.yaml`), and the new `error_resolution.json` (a real sfizz build refused because a track routed an unmapped instrument through the `solo` palette). `doctor_fixture_shape_holds` now asserts the recorded version satisfies `TESTED_SCOREKIT_RANGE` instead of pinning a literal, so the recording cannot rot behind the range again; `meta_fixture_shape_holds` asserts every meta track carries `id`/`palette`/`articulation` and that `instrument_resolution` is keyed by the same `track_id`. +- End-to-end sfizz routing proved with the exact argument vector `BuildParams::to_args` emits: `scorekit build routed.yaml -o routed.ogg --renderer sfizz --orchestration /Volumes/WT9/ScoreData/profiles/orchestrations/hybrid-cinematic.yaml --stems --json` produced a seamless 441000-sample loop, a `meta.json` whose `orchestration` block lists all four palettes, and stems named by track ID (`01-harmony.ogg`, `02-solo_line.ogg`) — a `palette: solo` track rendered through `scoredata-chamber` while the default track rendered through `scoredata-open`. `scorekit orchestration check` passed the same profile. +- Failure path confirmed rather than assumed: under scorekit's default `conservative` fallback mode an instrument outside the resolved palette aborts the build (exit 2, `code: "resolution"`), which is what scorebench's pre-build compatibility warning claims. +- `BenchError::Scorekit` now carries scorekit's `report` verbatim (boxed, so the rare payload does not widen every `Result`), and `errorText` folds it into one line per unresolved track: track ID, palette, leaf renderer profile, requested instrument, and the closest candidate with the reason it was rejected. Previously the agent and the panel saw only "1 unresolved instrument(s)". +- The scene preview table now shows each track's stable `id` and resolved `palette` next to its instrument, and a section segment lists the track IDs it mutes — without them a `mute: [pulse]` list could not be mapped back to a row after 0.5 replaced positional indices with IDs. Palette-less tracks use `OrchestrationCompat.tracks[].palette`, not a hard-coded `default` label, so an orchestration whose `default_palette` is `solo` is displayed truthfully. +- A 1440×900 headless browser smoke preloaded the real app with mocked Tauri IPC, opened a three-track project, and navigated to Preview. Two palette-less tracks routed by `default_palette: solo` displayed `solo`, the explicit `electronic` track displayed `electronic`, and section mute IDs remained legible without table or panel overflow. This exercises the Svelte render path; it is not a substitute for the native WebView smoke below. +- `manifest.rs` gained `leaf_profiles_resolve_against_the_orchestration_directory`, covering the layout scorekit's own sound-library contract prescribes: an absolute out-of-project orchestration whose palettes bind to `../renderers/.yaml`. Its temp-root helper also gained the atomic counter the other test modules use; without it parallel tests could collide on one directory and fail with `DirectoryNotEmpty`, which reproduced three times in five runs before the fix and zero times in five runs after. +- Gates after the change: `cargo fmt --check` clean; `cargo clippy --all-targets --all-features -- -D warnings` clean; `cargo test` passed 126 unit tests + the `tauri_boundary` integration test; `npm run check` reported 0 errors and 0 warnings; `npm test` passed 12 Vitest files (145 tests); `npm run build` succeeded and `npm run check:bundle` passed at 923.7 KiB raw / 270.0 KiB gzip client JavaScript. +- Not covered here: the native macOS GUI smoke for the new preview columns was not rerun in this pass. + +## ScoreKit 0.6 Agent arrangement contract (2026-07-26) + +This pass was verified against `/Users/wangjuntao/GitHub/scorekit/target/debug/scorekit` at ScoreKit `0.6.0` (repository `main` commit `c2e0b0c`; its CLI contract matches the `v0.6.0` release commit pinned in CI). The PATH-installed `~/.local/bin/scorekit` remained at 0.5.0, so every live 0.6 command used `SCOREBENCH_SCOREKIT` explicitly. + +- The supported range is `>=0.6.0, <0.7.0`. `doctor.json` and the real Forest build's `forest.meta.json` were re-recorded from 0.6.0; instrument-resolution reasons now record `general_midi_exact`. +- CI has a separate non-skippable contract job. It checks out the immutable ScoreKit v0.6.0 release commit, installs FFmpeg plus FluidSynth so `doctor` can prove both decode and render readiness, builds ScoreKit, sets `SCOREBENCH_REQUIRE_SCOREKIT_CONTRACT=1`, then runs the real v0.6 capability test inside the eleven-tool Agent surface. Missing, unready, or incompatible ScoreKit fails this job rather than turning the integration test into a no-op. +- The live contract test exercises `inspect-instruments`, exact texture match, exit-0 `no_match`, and `texture check` against a generated structured profile and audible WAV. It also proves `erhu` remains unresolved without an exact source even under `fallback_mode: flexible`, preserving ScoreKit's world-identity boundary. +- Scorebench now accepts both path-only and v0.6 structured texture bindings. The local, range-bound v0.6 preflight mirrors ScoreKit's required metadata, closed category/mode vocabulary, token/provenance constraints, and playback invariants; malformed structured profiles block readiness. A valid source used as `loop` when it only declares `one_shot` is reported before build in Rust, Agent tool output, and both observation-panel locations. +- The composing prompt contains a 1.0.0, ScoreKit-range-bound Arrangement Canon derived from the repository skill, alongside a live capability section and the independent active StylePack. Tests cover the intent contract, palette-inertia rule, motif/voice/rest/curve/loop constraints, exact-only world identities, source discovery/certification, and ordered G1–G5 gates. +- `write_scene.validate` was removed. Every scene write validates, and a request-local terminal gate tracks canonical project-relative identities. Failed `validate`, `build`, instrument-resolution, and grammar checks create independent blockers that only the corresponding successful retry can clear. A successful build made with one-shot overrides also rechecks the active `bench.json` configuration, so it cannot hide the failure the next default build would hit. Invalid or incompatible scenes may remain on disk for repair, but normal completion stays blocked. +- Text deltas still stream immediately, but the frontend marks them as a provisional draft. A turn is accepted only after all its tool results are known and the gate is clear, and remains transactional until the completed run history is durable on disk. Incomplete Responses, blocked/failed tools, cancellation, max-turn exhaustion, a later-turn failure, or a persistence failure discard or roll back the affected model claims. Channel events are scoped to one run/root/session, project switching is disabled while a run is live, and Stop targets the captured run rather than mutable UI selection. Scripted tests cover multi-delta streaming, incomplete output, failed preflights, text-plus-invalid-write, failed repair, max-turn rollback, durable finalization, persistence rollback, explicit-build override safety, `./`/symlink aliases, Unix backslash filename non-collision, and stale frontend run identities. +- The offline capability fixture produced one pass and one fail across tempo, track count, melody-rest ratio, mode, loop, and motif-contour assertions (`pass_rate: 0.5`). `--min-pass-rate 0.5` exited 0 and `0.75` exited 1. +- Adding the `capability_eval` helper creates a second Cargo binary, so `Cargo.toml` explicitly sets `default-run = "scorebench"`. The exact Tauri dev command `cargo run --no-default-features --color always --` resolves to `target/debug/scorebench`; CI checks the metadata field so a future helper binary cannot silently break development startup again. +- Final gates: `cargo fmt --all -- --check`, `cargo clippy --all-targets --all-features -- -D warnings`, and the required explicit-v0.6 `cargo test --all-features` passed (151 Rust unit tests plus the Tauri-boundary integration test). `npm run check` reported 0 errors and 0 warnings; `npm test` passed 12 files / 149 tests; `npm run build` succeeded; `npm run check:bundle` passed at 926.4 KiB raw / 270.9 KiB gzip; `actionlint .github/workflows/*.yml` and `mdbook build docs-site` passed. +- Not covered here: no live LLM quality run or native macOS GUI smoke was performed. The opt-in M8 capability runner measures saved output samples without making live model calls; statistical model/prompt baselines and signal-level V-A-T assertions remain follow-up acceptance work. diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml index 2cfa6cc..7dd26ce 100644 --- a/src-tauri/Cargo.toml +++ b/src-tauri/Cargo.toml @@ -6,6 +6,7 @@ authors = ["talkincode"] license = "MIT" repository = "https://github.com/talkincode/scorebench" edition = "2021" +default-run = "scorebench" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html diff --git a/src-tauri/arrangement/canon.yaml b/src-tauri/arrangement/canon.yaml new file mode 100644 index 0000000..a141e4f --- /dev/null +++ b/src-tauri/arrangement/canon.yaml @@ -0,0 +1,130 @@ +schema_version: 1 +id: scorekit-arrangement-canon +version: 1.0.0 +source_scorekit_range: ">=0.6.0, <0.7.0" +provenance: + repository: talkincode/scorekit + commit: c2e0b0c + references: + - skills/scorekit/SKILL.md + - skills/scorekit/audit.md + - skills/scorekit/palettes.md + +authority: + live_schema: + scene: "scorekit schema --json" + grammar: "scorekit schema --grammar --json" + texture_profile: "scorekit schema --texture-profile --json" + renderer_profile: "scorekit schema --profile --json" + orchestration: "scorekit schema --orchestration --json" + rule: "Treat the live command output as the complete writable vocabulary." + prohibition: "Use only live-schema fields and values; never invent a scene field, instrument, palette, articulation, or texture source." + narrative_boundary: "story is informational; translate narrative into schema-legal musical decisions." + +intent_contract: + before_write: + state: "One concise, falsifiable translation of the user intent." + expected_changes: + - "Name the scene fields expected to change and their direction or target." + - "Name what must remain invariant: motif identity, loop function, style, or requested source." + - "Call out any requested result the live schema cannot express before editing." + after_write: + compare: + - "Report whether validation and semantic diff match the expected changes." + - "Report source-resolution and build evidence without claiming unmeasured musical quality." + - "Repair or explicitly explain every mismatch before declaring completion." + +palette_inertia: + timing: "Complete this decision before writing tracks." + name_the_inertia_answer: "State the habitual palette, key/mode, tempo, and meter that first came to mind." + candidates: + candidate_count: "2..=3" + compare_by: [setting, era, technology, culture, character, function] + mood_only_is_not_a_reason: true + selection: "Pick one candidate and cite brief evidence; name why the rejected candidates fit less well." + orchestral_rule: "Orchestral strings must win this comparison; they never inherit by default." + consecutive_deliverables: + minimum_changed_axes: 3 + axes: [palette_family, key_mode, tempo_class, meter, lead_timbre, pulse, harmony_color, density] + exception: "A user-pinned matching or series style; state that override." + +craft: + motif_economy: + core_identity_count: 1 + total_motifs_max: 2 + core_cell_notes: "4..=6" + valid_variations: [octave, rhythm, answering_instrument] + rule: "Restate and develop the identity instead of piling up unrelated tunes." + voice_discipline: + simultaneous_non_rest_melody_voices_max: 2 + exception: "One brief-justified tutti corner; disclose it in the audit." + breathing: + rest_ratio_per_melody_min: 0.30 + rest_degree: 0 + long_rest_note_beats_max: 16 + rule: "Split longer silence into multiple legal rest notes; silence is arrangement material." + role_coverage: + roles: [lead, harmony, foundation, pulse] + rule: "No unintended gap and no duplicate role without registral or rhythmic contrast." + entrances_and_exits: + whole_span_patterns: [sustain, arpeggio, bass, drums, tabla] + rule: "For a mid-span entrance or exit, use a melody track with explicit rests or a section mute by stable track id." + emotional_curve: + measure: "intensity x register x active non-rest voice count over time" + shape: "rise -> peak -> cut_or_contrast -> return" + reject: "A flat plateau or a peak that never releases." + register_spacing: + rule: "Keep one clear low anchor, at most two concurrent voices per band, and the lead clear of the harmonic bed." + loop_and_ending: + loop: + rule: "The dynamics end returns to its start, and the last harmony must pull toward the first." + evidence: "meta.json loop_samples and total_samples are sane; listen across repeated seams." + one_shot: + rule: "The final cadence and tail match the declared intent." + determinism: + rule: "If performance.humanize is present, set its seed; identical decisions must remain reproducible." + +world_identity: + identities: [erhu, pipa, guzheng, dizi, shakuhachi, shamisen, sitar, tabla, oud, ney, duduk] + fallback_policy: exact_only + rule: "Never substitute into, out of, or within this family." + gm_exact_melodic: [shakuhachi, shamisen, sitar] + renderer_profile_required_melodic: [erhu, pipa, guzheng, dizi, oud, ney, duduk] + percussion_exception: "tabla requires pattern tabla; keep the identity exact and verify the active render path." + promise_gate: "Do not promise a world identity until inspect-instruments proves an exact usable source, or visibly re-orchestrate." + +textures: + source_honesty: "Never guess textures[].source; scene values are portable profile keys, never file paths." + discover: + enumerate: "scorekit --json texture inspect " + filter: "Use exact category/tag/use-case filters; repeated tags are conjunctive." + per_source: "scorekit --json texture inspect --source " + no_match: "no_match means re-orchestrate or add a real source; never choose a nearest-looking name." + certify: "scorekit --json texture check " + schedule_rule: "The scene mode must be declared in the selected source's playback.modes." + runtime_boundary: "Distance, weather, RPM, and other world-reactive behavior belongs in the game engine." + +delivery_gates: + preflight: + doctor: "scorekit --json doctor must report a ready toolchain." + orchestration: "When configured, scorekit orchestration check must pass." + ordered: + G1_validate: + command: "scorekit --json validate " + pass: "exit 0; repair every live-schema error." + G2_lint: + command: "scorekit --json lint --grammar " + pass: "exit 0 for every declared project grammar; n/a only when none exists." + G3_inspect_instruments: + command: "scorekit --json inspect-instruments [--orchestration ]" + pass: "No missing or rejected source. A fallback requires a visible musical review, mapping fix, or re-orchestration; never weaken resolver policy to silence it." + G4_textures: + command: "Run texture inspect for every declared source, then texture check for the active profile." + pass: "Every source exists, permits the requested mode, decodes, and certifies audible; n/a only when the scene has no textures." + G5_build_evidence: + command: "scorekit build -o with the active orchestration, texture profile, and requested stems." + pass: "Build exits 0; meta.json resolution is clean or reviewed, loop metadata is sane, requested stems are listed, and story is echoed when declared." + completion: + failed_blocking_gate: do_not_report_completion + craft_findings: "Fix each finding or give one concise, brief-grounded justification." + verdicts: [BLOCKED, SHIP_WITH_NOTES, CLEAN] diff --git a/src-tauri/src/agent.rs b/src-tauri/src/agent.rs index 400db15..cbc2133 100644 --- a/src-tauri/src/agent.rs +++ b/src-tauri/src/agent.rs @@ -18,8 +18,8 @@ use tokio_util::sync::CancellationToken; use crate::error::BenchError; use crate::llm::types::{InputItem, InputRole, MessageContent, ResponseEvent, ResponsesRequest}; use crate::llm::{ResponseStream, ResponsesClient}; -use crate::{manifest, memory, project, scorekit, styles}; -use tools::ToolBelt; +use crate::{arrangement, manifest, memory, project, scorekit, styles}; +use tools::{SceneGateKind, SceneGateUpdate, ToolBelt, ToolResult}; const TOOL_OUTPUT_LIMIT: usize = 64 * 1024; @@ -29,6 +29,16 @@ pub enum AgentEvent { Text { text: String, }, + /// Accepts the current streamed draft into this in-flight run. It remains + /// transactional in the frontend until `TextFinalize` follows persistence. + TextCommit, + /// Drops the preceding streamed draft because a tool failed or a scene + /// validation gate remains active. + TextDiscard, + /// Confirms that every accepted draft from this run is now on disk. + TextFinalize, + /// Removes every accepted/provisional draft from a run that later failed. + TextRollback, Warning { text: String, }, @@ -69,6 +79,38 @@ impl AgentTransport for ResponsesClient { } } +pub trait ToolExecutor: Sync { + fn execute<'a>( + &'a self, + call: crate::llm::types::FunctionCall, + ) -> Pin> + Send + 'a>>; + + fn failure_gates( + &self, + call: &crate::llm::types::FunctionCall, + error: &BenchError, + ) -> Vec { + tools::failure_gates(call, error) + } +} + +impl ToolExecutor for ToolBelt { + fn execute<'a>( + &'a self, + call: crate::llm::types::FunctionCall, + ) -> Pin> + Send + 'a>> { + Box::pin(async move { ToolBelt::execute(self, call).await }) + } + + fn failure_gates( + &self, + call: &crate::llm::types::FunctionCall, + error: &BenchError, + ) -> Vec { + ToolBelt::failure_gates(self, call, error) + } +} + type SessionKey = (PathBuf, String); #[derive(Default)] @@ -233,18 +275,54 @@ pub fn system_prompt( }; let style_section = style.map(styles::prompt_section).unwrap_or_default(); let render_section = render_config_section(root); + let arrangement_section = arrangement::prompt_section(); + let capability_section = scorekit_capability_section(); Ok(format!( "You are scorebench, the composing agent for one scorekit project.\n\ You are the only writer of scene YAML. Never invent an editing UI or render audio yourself.\n\ Use the provided tools; scorekit validation errors are authoritative.\n\ Keep paths project-relative. write_scene validates automatically and reports the result;\n\ - fix any reported problem before building and explain musical decisions concisely.\n\n\ + fix any reported problem before building and explain musical decisions concisely.\n\ + Authority order: live ScoreKit schema and machine failures are mandatory; explicit user \ + constraints govern the musical result; the Arrangement Canon supplies cross-style craft \ + and audit rules; the active StylePack narrows stylistic choices. Disclose and justify any \ + user-requested exception to a non-machine Canon rule.\n\ + For textures, inspect exact catalog matches before writing source keys; no_match never authorizes invention.\n\ + For world instruments and non-default palettes, inspect instrument resolution before promising or building.\n\n\ + {capability_section}{arrangement_section}\n\ CURRENT PROJECT SNAPSHOT:\n{snapshot}\n\n{render_section}{style_section}\ ROLLING PROJECT MEMORY:\n{project_memory}\n\n\ SCOREKIT SCENE JSON SCHEMA:\n{schema}" )) } +fn scorekit_capability_section() -> String { + let handshake = scorekit::handshake(); + let live_version = handshake.version.as_deref().unwrap_or("(unavailable)"); + let compatibility = match handshake.compatible { + Some(true) => "compatible", + Some(false) => "outside-tested-range", + None => "unverified", + }; + let warning = handshake + .warning + .as_deref() + .map(|warning| format!("\nwarning: {warning}")) + .unwrap_or_default(); + format!( + "LIVE SCOREKIT CAPABILITY:\n\ + version: {live_version}\n\ + tested range: {}\n\ + toolchain ready: {}\n\ + compatibility: {compatibility}{warning}\n\ + deterministic read-only preflights: inspect_instruments, inspect_textures, check_texture_profile\n\ + General MIDI exact world identities: shakuhachi, shamisen, sitar.\n\ + erhu, pipa, guzheng, dizi, tabla, oud, ney, and duduk require an exact active source; never rely on fallback.\n\n", + scorekit::TESTED_SCOREKIT_RANGE, + handshake.ready + ) +} + /// Prompt block describing the render configuration persisted in bench.json, /// including the instrument and texture source keys active profiles map, so /// the model only composes material the configured toolchain can build. @@ -254,22 +332,48 @@ fn render_config_section(root: &Path) -> String { }; let renderer = render.renderer.as_deref().unwrap_or("(default)"); let mut section = format!("ACTIVE RENDER CONFIGURATION (bench.json):\nrenderer: {renderer}\n"); - match (render.renderer.as_deref(), render.profile.as_deref()) { - (Some("sfizz"), Some(profile)) if !profile.trim().is_empty() => { - match manifest::profile_instruments(root, profile) { - Ok((name, instruments)) => { - let name = name.unwrap_or_else(|| profile.to_owned()); + match (render.renderer.as_deref(), render.orchestration.as_deref()) { + (Some("sfizz"), Some(orchestration)) if !orchestration.trim().is_empty() => { + match manifest::load_orchestration(root, orchestration) { + Ok(info) => { section.push_str(&format!( - "profile: {profile} ({name})\n\ - instruments mapped by this profile: {}\n\ - Any track instrument outside this list will FAIL the sfizz build.\n\ - Compose only with mapped instruments, or tell the user which mapping is missing.\n", - instruments.join(", ") + "orchestration: {orchestration}{}\n\ + default palette: {}\n", + info.name + .as_deref() + .map(|name| format!(" ({name})")) + .unwrap_or_default(), + info.default_palette.as_deref().unwrap_or("(none)"), )); + for palette in info.palettes() { + if let Some(error) = &palette.error { + section.push_str(&format!( + "palette `{}` -> {} : WARNING unusable ({error}); builds routed through it will fail.\n", + palette.name, palette.declared_profile + )); + } else { + let name = palette + .profile_name + .as_deref() + .unwrap_or(&palette.declared_profile); + section.push_str(&format!( + "palette `{}` -> {} ({name}): instruments mapped: {}\n", + palette.name, + palette.declared_profile, + palette.instruments.join(", ") + )); + } + } + section.push_str( + "Give each track a stable `id` and, when it should use a palette other \ + than the default, a `palette` matching one listed above. A track \ + instrument outside its resolved palette's mapped list will FAIL the \ + sfizz build.\n", + ); } Err(error) => { section.push_str(&format!( - "profile: {profile}\nWARNING: the profile could not be read ({error}); builds will fail until it is fixed.\n" + "orchestration: {orchestration}\nWARNING: the orchestration could not be read ({error}); builds will fail until it is fixed.\n" )); } } @@ -284,11 +388,21 @@ fn render_config_section(root: &Path) -> String { match manifest::texture_profile_sources(root, profile) { Ok((name, sources)) => { let name = name.unwrap_or_else(|| profile.to_owned()); + let catalog = if sources.len() <= 12 { + sources.join(", ") + } else { + format!( + "{} source keys (omitted from the prompt; query them with inspect_textures)", + sources.len() + ) + }; section.push_str(&format!( "texture profile: {profile} ({name})\n\ texture sources mapped by this profile: {}\n\ - Use only these portable source keys in scene textures; any other key will FAIL the build.\n", - sources.join(", ") + Names alone are not selection evidence: call inspect_textures with exact filters \ + before writing a source, and honor its declared playback modes. Any unmapped key \ + or unsupported mode will FAIL the build.\n", + catalog )); } Err(error) => { @@ -306,6 +420,14 @@ fn render_config_section(root: &Path) -> String { pub struct RunOutcome { pub history: Vec, pub prompt_tokens: Option, + pub status: RunStatus, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RunStatus { + Complete, + MaxTurns, + Blocked, } pub async fn run_project( @@ -330,9 +452,9 @@ pub async fn run_project( .await } -pub async fn run_loop( +pub async fn run_loop( transport: &T, - tool_belt: &ToolBelt, + tool_belt: &E, instructions: String, mut input: Vec, max_turns: u32, @@ -340,14 +462,23 @@ pub async fn run_loop( mut emit: impl FnMut(AgentEvent), ) -> Result { let mut prompt_tokens = None; + let mut scene_gate = HashMap::::new(); for _ in 0..max_turns { if cancellation.is_cancelled() { return Err(BenchError::cancelled()); } + let gate_was_active = !scene_gate.is_empty(); + let mut request_input = input.clone(); + if gate_was_active { + request_input.push(InputItem::Message { + role: InputRole::User, + content: validation_gate_message(&scene_gate).into(), + }); + } let request = ResponsesRequest { model: String::new(), instructions: Some(instructions.clone()), - input: input.clone(), + input: request_input, tools: tools::definitions(), max_output_tokens: None, stream: true, @@ -359,7 +490,16 @@ pub async fn run_loop( let mut terminal = false; while let Some(event) = stream.next().await { - match event? { + let event = match event { + Ok(event) => event, + Err(error) => { + if !text.is_empty() { + emit(AgentEvent::TextDiscard); + } + return Err(error); + } + }; + match event { ResponseEvent::OutputTextDelta { delta, .. } => { text.push_str(&delta); emit(AgentEvent::Text { text: delta }); @@ -375,16 +515,22 @@ pub async fn run_loop( } } ResponseEvent::Incomplete { reason, .. } => { - terminal = true; - emit(AgentEvent::Warning { - text: format!( - "response truncated ({})", + if !text.is_empty() { + emit(AgentEvent::TextDiscard); + } + return Err(BenchError::agent( + "response_incomplete", + format!( + "response was incomplete ({})", reason.as_deref().unwrap_or("unknown reason") ), - }); + )); } ResponseEvent::Failed { code, message, .. } | ResponseEvent::Error { code, message } => { + if !text.is_empty() { + emit(AgentEvent::TextDiscard); + } return Err(BenchError::Llm { message, status: None, @@ -396,26 +542,52 @@ pub async fn run_loop( } } if !terminal { + if !text.is_empty() { + emit(AgentEvent::TextDiscard); + } return Err(BenchError::llm( "Responses stream ended before a terminal event", )); } - if !text.is_empty() { - input.push(InputItem::Message { - role: InputRole::Assistant, - content: text.into(), - }); - } if calls.is_empty() { + if !scene_gate.is_empty() { + if !text.is_empty() { + emit(AgentEvent::TextDiscard); + } + emit(AgentEvent::Warning { + text: format!( + "Agent completion withheld: {}. Repair and revalidate every listed scene before finishing.", + gate_summary(&scene_gate) + ), + }); + continue; + } + if !text.is_empty() { + emit(AgentEvent::TextCommit); + input.push(InputItem::Message { + role: InputRole::Assistant, + content: text.into(), + }); + } return Ok(RunOutcome { history: input, prompt_tokens, + status: RunStatus::Complete, }); } + // A model may emit a success claim and a tool call in the same + // response. Deltas remain an explicitly unverified UI draft until + // every tool result is known; otherwise an invalid write or failed + // repair could be committed before it opens (or preserves) the gate. + let assistant_text_index = input.len(); + let mut all_tools_succeeded = true; for call in calls { if cancellation.is_cancelled() { + if !text.is_empty() { + emit(AgentEvent::TextDiscard); + } return Err(BenchError::cancelled()); } input.push(InputItem::FunctionCall { @@ -429,6 +601,9 @@ pub async fn run_loop( }); match tool_belt.execute(call.clone()).await { Ok(result) => { + for update in result.scene_gates { + apply_gate_update(&mut scene_gate, update); + } emit(AgentEvent::ToolOk { name: call.name, summary: result.summary, @@ -440,6 +615,10 @@ pub async fn run_loop( }); } Err(error) => { + all_tools_succeeded = false; + for update in tool_belt.failure_gates(&call, &error) { + apply_gate_update(&mut scene_gate, update); + } emit(AgentEvent::ToolErr { name: call.name, error: error.clone(), @@ -452,8 +631,37 @@ pub async fn run_loop( } } } + if cancellation.is_cancelled() { + if !text.is_empty() { + emit(AgentEvent::TextDiscard); + } + return Err(BenchError::cancelled()); + } + if !text.is_empty() && all_tools_succeeded && scene_gate.is_empty() { + emit(AgentEvent::TextCommit); + input.insert( + assistant_text_index, + InputItem::Message { + role: InputRole::Assistant, + content: text.into(), + }, + ); + } else if !text.is_empty() { + emit(AgentEvent::TextDiscard); + } } + let status = if scene_gate.is_empty() { + RunStatus::MaxTurns + } else { + emit(AgentEvent::Warning { + text: format!( + "Validation gate remains blocked after {max_turns} model turns: {}.", + gate_summary(&scene_gate) + ), + }); + RunStatus::Blocked + }; emit(AgentEvent::Warning { text: format!( "Agent stopped after the configured {max_turns} model turns. Increase the limit in Settings or continue with a new message." @@ -462,9 +670,63 @@ pub async fn run_loop( Ok(RunOutcome { history: input, prompt_tokens, + status, }) } +fn apply_gate_update(scene_gate: &mut HashMap, update: SceneGateUpdate) { + let path = scene_gate_key(&update.path); + let (key, label) = match update.kind { + SceneGateKind::Readiness => (format!("readiness:{path}"), format!("`{path}` readiness")), + SceneGateKind::Build => (format!("build:{path}"), format!("`{path}` build")), + SceneGateKind::InstrumentResolution => ( + format!("instrument-resolution:{path}"), + format!("`{path}` instrument resolution"), + ), + SceneGateKind::Grammar { grammar } => { + let grammar = scene_gate_key(&grammar); + ( + format!("grammar:{grammar}:{path}"), + format!("`{path}` grammar `{grammar}`"), + ) + } + }; + if update.ready { + scene_gate.remove(&key); + } else { + scene_gate.insert(key, format!("{label} ({})", update.reason)); + } +} + +fn validation_gate_message(scene_gate: &HashMap) -> String { + format!( + "[SCOREBENCH VALIDATION GATE]\nNormal completion is blocked for: {}.\n\ + Repair the scene or configuration, then re-run each named check. Do not claim success while any blocker remains.", + gate_summary(scene_gate) + ) +} + +fn gate_summary(scene_gate: &HashMap) -> String { + let mut entries = scene_gate.values().cloned().collect::>(); + entries.sort_unstable(); + entries.join("; ") +} + +fn scene_gate_key(path: &str) -> String { + let mut normalized = PathBuf::new(); + for component in Path::new(path).components() { + match component { + std::path::Component::CurDir => {} + std::path::Component::Normal(part) => normalized.push(part), + // Tool results originate from the project-confined ToolBelt. Keep + // unexpected components visible rather than silently resolving a + // path outside that trust boundary. + _ => return path.to_owned(), + } + } + normalized.to_string_lossy().into_owned() +} + pub async fn compact_project( transport: &T, root: &Path, @@ -597,23 +859,30 @@ mod tests { struct ScriptedTransport { turns: Mutex, BenchError>>>, + requests: Mutex>, } impl ScriptedTransport { fn new(turns: Vec, BenchError>>) -> Self { Self { turns: Mutex::new(turns.into()), + requests: Mutex::new(Vec::new()), } } + + fn requests(&self) -> Vec { + self.requests.lock().unwrap().clone() + } } impl AgentTransport for ScriptedTransport { fn stream<'a>( &'a self, - _request: ResponsesRequest, + request: ResponsesRequest, _cancellation: CancellationToken, ) -> Pin> + Send + 'a>> { Box::pin(async move { + self.requests.lock().unwrap().push(request); match self.turns.lock().unwrap().pop_front().unwrap() { Ok(events) => { Ok(Box::pin(stream::iter(events.into_iter().map(Ok))) as ResponseStream) @@ -624,6 +893,55 @@ mod tests { } } + struct ScriptedTools { + results: Mutex>>, + } + + impl ScriptedTools { + fn new(results: Vec>) -> Self { + Self { + results: Mutex::new(results.into()), + } + } + } + + impl ToolExecutor for ScriptedTools { + fn execute<'a>( + &'a self, + _call: FunctionCall, + ) -> Pin> + Send + 'a>> { + Box::pin(async move { self.results.lock().unwrap().pop_front().unwrap() }) + } + } + + fn scripted_tool_result(path: &str, ready: bool, reason: &str) -> ToolResult { + ToolResult { + output: serde_json::json!({ + "ok": true, + "path": path, + "validation": {"status": if ready { "valid" } else { "invalid" }} + }) + .to_string(), + summary: reason.into(), + detail: None, + scene_gates: vec![tools::SceneGateUpdate { + kind: tools::SceneGateKind::Readiness, + path: path.into(), + ready, + reason: reason.into(), + }], + } + } + + fn scripted_plain_tool_result(reason: &str) -> ToolResult { + ToolResult { + output: serde_json::json!({"ok": true}).to_string(), + summary: reason.into(), + detail: None, + scene_gates: Vec::new(), + } + } + fn completed() -> ResponseEvent { ResponseEvent::Completed { response_id: Some("response_fixture".into()), @@ -661,10 +979,163 @@ mod tests { root } + #[test] + #[cfg(unix)] + fn scene_gate_keys_preserve_distinct_unix_backslash_filenames() { + assert_ne!( + scene_gate_key(r"dir\scene.yaml"), + scene_gate_key("dir/scene.yaml") + ); + assert_eq!(scene_gate_key("./scene.yaml"), scene_gate_key("scene.yaml")); + } + + #[tokio::test] + async fn plain_text_remains_streamed_as_individual_deltas() { + let transport = ScriptedTransport::new(vec![Ok(vec![ + ResponseEvent::OutputTextDelta { + item_id: None, + output_index: Some(0), + delta: "First".into(), + }, + ResponseEvent::OutputTextDelta { + item_id: None, + output_index: Some(0), + delta: " second".into(), + }, + completed(), + ])]); + let tools = ScriptedTools::new(vec![]); + let mut events = Vec::new(); + let outcome = run_loop( + &transport, + &tools, + "fixture prompt".into(), + vec![], + 1, + CancellationToken::new(), + |event| events.push(event), + ) + .await + .unwrap(); + + assert_eq!(outcome.status, RunStatus::Complete); + let deltas = events + .iter() + .filter_map(|event| match event { + AgentEvent::Text { text } => Some(text.as_str()), + _ => None, + }) + .collect::>(); + assert_eq!(deltas, vec!["First", " second"]); + assert!(matches!(events.last(), Some(AgentEvent::TextCommit))); + assert!(matches!( + outcome.history.last(), + Some(InputItem::Message { + role: InputRole::Assistant, + content, + }) if content.display_text() == "First second" + )); + } + #[tokio::test] - async fn scripted_loop_writes_scene_then_finishes() { + async fn incomplete_response_discards_streamed_text_and_fails() { + let transport = ScriptedTransport::new(vec![Ok(vec![ + ResponseEvent::OutputTextDelta { + item_id: None, + output_index: Some(0), + delta: "Cut off mid-sentence".into(), + }, + ResponseEvent::Incomplete { + response_id: Some("response_fixture".into()), + reason: Some("max_output_tokens".into()), + }, + ])]); + let tools = ScriptedTools::new(vec![]); + let mut events = Vec::new(); + let error = run_loop( + &transport, + &tools, + "fixture prompt".into(), + vec![], + 1, + CancellationToken::new(), + |event| events.push(event), + ) + .await + .unwrap_err(); + + assert!(matches!( + error, + BenchError::Agent { ref code, .. } if code == "response_incomplete" + )); + assert!(events + .iter() + .any(|event| matches!(event, AgentEvent::TextDiscard))); + assert!(!events + .iter() + .any(|event| matches!(event, AgentEvent::TextCommit))); + } + + #[tokio::test] + async fn failed_scene_preflight_blocks_a_later_text_only_completion() { let transport = ScriptedTransport::new(vec![ Ok(vec![ + call("validate_scene", r#"{"path":"broken.yaml"}"#), + completed(), + ]), + Ok(vec![ + ResponseEvent::OutputTextDelta { + item_id: None, + output_index: Some(0), + delta: "Everything is complete.".into(), + }, + completed(), + ]), + ]); + let tools = ScriptedTools::new(vec![Err(BenchError::agent( + "fixture_validation", + "scene is invalid", + ))]); + let mut events = Vec::new(); + let outcome = run_loop( + &transport, + &tools, + "fixture prompt".into(), + vec![], + 2, + CancellationToken::new(), + |event| events.push(event), + ) + .await + .unwrap(); + + assert_eq!(outcome.status, RunStatus::Blocked); + let requests = transport.requests(); + assert!(requests[1].input.iter().any(|item| matches!( + item, + InputItem::Message { + role: InputRole::User, + content, + } if content.display_text().contains("SCOREBENCH VALIDATION GATE") + && content.display_text().contains("broken.yaml") + ))); + assert!(events + .iter() + .any(|event| matches!(event, AgentEvent::TextDiscard))); + assert!(!events + .iter() + .any(|event| matches!(event, AgentEvent::TextCommit))); + } + + #[tokio::test] + async fn invalid_scene_cannot_finish_as_a_successful_turn() { + let transport = ScriptedTransport::new(vec![ + Ok(vec![ + ResponseEvent::OutputTextDelta { + item_id: None, + output_index: Some(0), + delta: "Scene written.".into(), + }, call( "write_scene", r#"{"path":"forest.yaml","content":"title: Forest\nbars: 8\n"}"#, @@ -691,26 +1162,198 @@ mod tests { role: InputRole::User, content: "write a scene".into(), }], - 4, + 2, CancellationToken::new(), |event| events.push(event), ) .await .unwrap(); + assert_eq!(outcome.status, RunStatus::Blocked); assert!(std::fs::read_to_string(root.join("forest.yaml")) .unwrap() .contains("Forest")); assert!(events.iter().any( |event| matches!(event, AgentEvent::ToolOk { name, .. } if name == "write_scene") )); + assert!(events.iter().any(|event| { + matches!( + event, + AgentEvent::Warning { text } + if text.contains("completion withheld") + && text.contains("forest.yaml") + ) + })); + assert!( + outcome.history.iter().all(|item| { + !matches!( + item, + InputItem::Message { + role: InputRole::User, + content, + } if content.display_text().contains("SCOREBENCH VALIDATION GATE") + ) + }), + "the internal gate reminder must be ephemeral, not transcript history" + ); + assert!(events + .iter() + .any(|event| matches!(event, AgentEvent::Text { text } if text == "Scene written."))); + assert!(!events + .iter() + .any(|event| matches!(event, AgentEvent::TextCommit))); + assert_eq!( + events + .iter() + .filter(|event| matches!(event, AgentEvent::TextDiscard)) + .count(), + 2, + "both the invalid tool-turn draft and blocked completion draft are discarded" + ); + assert!( + outcome.history.iter().all(|item| !matches!( + item, + InputItem::Message { content, .. } + if content.display_text() == "Scene written." + )), + "a blocked completion claim must not enter persistent history" + ); + std::fs::remove_dir_all(root).unwrap(); + } + + #[tokio::test] + async fn gate_remains_ephemeral_until_the_same_scene_is_repaired() { + let transport = ScriptedTransport::new(vec![ + Ok(vec![ + call("write_scene", r#"{"path":"forest.yaml","content":"bad"}"#), + completed(), + ]), + Ok(vec![ + ResponseEvent::OutputTextDelta { + item_id: None, + output_index: Some(0), + delta: "Fixed too early.".into(), + }, + call( + "write_scene", + r#"{"path":"./forest.yaml","content":"still bad"}"#, + ), + completed(), + ]), + Ok(vec![ + ResponseEvent::OutputTextDelta { + item_id: None, + output_index: Some(0), + delta: "Finished too early.".into(), + }, + completed(), + ]), + Ok(vec![ + call( + "write_scene", + r#"{"path":"./forest.yaml","content":"repaired"}"#, + ), + completed(), + ]), + Ok(vec![ + ResponseEvent::OutputTextDelta { + item_id: None, + output_index: Some(0), + delta: "Validated and finished.".into(), + }, + completed(), + ]), + ]); + let tools = ScriptedTools::new(vec![ + Ok(scripted_tool_result("forest.yaml", false, "scene invalid")), + Ok(scripted_tool_result( + "./forest.yaml", + false, + "scene still invalid", + )), + Ok(scripted_tool_result("./forest.yaml", true, "scene valid")), + ]); + let mut events = Vec::new(); + let outcome = run_loop( + &transport, + &tools, + "fixture prompt".into(), + vec![], + 5, + CancellationToken::new(), + |event| events.push(event), + ) + .await + .unwrap(); + + assert_eq!(outcome.status, RunStatus::Complete); + let requests = transport.requests(); + assert_eq!(requests.len(), 5); + let has_gate = |request: &ResponsesRequest| { + request.input.iter().any(|item| { + matches!( + item, + InputItem::Message { + role: InputRole::User, + content, + } if content.display_text().contains("SCOREBENCH VALIDATION GATE") + ) + }) + }; + assert!(!has_gate(&requests[0])); + assert!(has_gate(&requests[1])); + assert!(has_gate(&requests[2])); + assert!(has_gate(&requests[3])); + assert!(!has_gate(&requests[4]), "repair must clear the path gate"); + assert!(events.iter().any( + |event| matches!(event, AgentEvent::Warning { text } if text.contains("completion withheld")) + )); + assert!(events + .iter() + .any(|event| matches!(event, AgentEvent::Text { text } if text == "Fixed too early."))); + assert!(events.iter().any( + |event| matches!(event, AgentEvent::Text { text } if text == "Finished too early.") + )); + assert_eq!( + events + .iter() + .filter(|event| matches!(event, AgentEvent::TextDiscard)) + .count(), + 2 + ); + assert_eq!( + events + .iter() + .filter(|event| matches!(event, AgentEvent::TextCommit)) + .count(), + 1 + ); + assert!( + outcome.history.iter().all(|item| { + !matches!( + item, + InputItem::Message { content, .. } + if content.display_text().contains("SCOREBENCH VALIDATION GATE") + ) + }), + "internal gate reminders are request-local" + ); + assert!(outcome.history.iter().all(|item| !matches!( + item, + InputItem::Message { content, .. } + if content.display_text() == "Fixed too early." + ))); + assert!(outcome.history.iter().all(|item| !matches!( + item, + InputItem::Message { content, .. } + if content.display_text() == "Finished too early." + ))); assert!(matches!( outcome.history.last(), Some(InputItem::Message { role: InputRole::Assistant, - .. - }) + content, + }) if content.display_text() == "Validated and finished." )); - std::fs::remove_dir_all(root).unwrap(); } #[tokio::test] @@ -729,7 +1372,7 @@ mod tests { let root = temp_project("unknown"); let belt = ToolBelt::new(root.clone()).unwrap(); let mut events = Vec::new(); - run_loop( + let outcome = run_loop( &transport, &belt, "fixture".into(), @@ -740,6 +1383,7 @@ mod tests { ) .await .unwrap(); + assert_eq!(outcome.status, RunStatus::Complete); assert!(events .iter() .any(|event| matches!(event, AgentEvent::ToolErr { .. }))); @@ -756,7 +1400,7 @@ mod tests { let root = temp_project("max-turns"); let belt = ToolBelt::new(root.clone()).unwrap(); let mut events = Vec::new(); - run_loop( + let outcome = run_loop( &transport, &belt, "fixture".into(), @@ -767,12 +1411,53 @@ mod tests { ) .await .unwrap(); + assert_eq!(outcome.status, RunStatus::MaxTurns); assert!(events .iter() .any(|event| matches!(event, AgentEvent::Warning { .. }))); std::fs::remove_dir_all(root).unwrap(); } + #[tokio::test] + async fn early_committed_text_is_not_a_complete_outcome_after_max_turns() { + let transport = ScriptedTransport::new(vec![ + Ok(vec![ + ResponseEvent::OutputTextDelta { + item_id: None, + output_index: Some(0), + delta: "I will inspect first.".into(), + }, + call("doctor", "{}"), + completed(), + ]), + Ok(vec![call("doctor", "{}"), completed()]), + ]); + let tools = ScriptedTools::new(vec![ + Ok(scripted_plain_tool_result("doctor ready")), + Ok(scripted_plain_tool_result("doctor ready")), + ]); + let mut events = Vec::new(); + let outcome = run_loop( + &transport, + &tools, + "fixture".into(), + vec![], + 2, + CancellationToken::new(), + |event| events.push(event), + ) + .await + .unwrap(); + + assert_eq!(outcome.status, RunStatus::MaxTurns); + assert!(events + .iter() + .any(|event| matches!(event, AgentEvent::TextCommit))); + assert!(events.iter().any( + |event| matches!(event, AgentEvent::Warning { text } if text.contains("2 model turns")) + )); + } + #[tokio::test] async fn transport_failure_stops_cleanly() { let transport = ScriptedTransport::new(vec![Err(BenchError::llm("offline"))]); @@ -946,6 +1631,28 @@ mod tests { std::fs::remove_dir_all(root).unwrap(); } + #[test] + fn system_prompt_separates_arrangement_canon_style_and_live_capability() { + let root = temp_project("arrangement-canon"); + let pack = styles::builtins().into_iter().next().unwrap(); + let prompt = system_prompt(&root, "main", Some(&pack)).unwrap(); + + assert!(prompt.contains("ARRANGEMENT CANON `scorekit-arrangement-canon` v1.0.0")); + assert!(prompt.contains("before_write")); + assert!(prompt.contains("name_the_inertia_answer")); + assert!(prompt.contains("Never guess textures[].source")); + assert!(prompt.contains("LIVE SCOREKIT CAPABILITY")); + assert!(prompt.contains(scorekit::TESTED_SCOREKIT_RANGE)); + assert!(prompt.contains("erhu, pipa, guzheng, dizi, tabla, oud, ney, and duduk")); + assert!(prompt.contains("Authority order: live ScoreKit schema")); + assert!(prompt.contains("ACTIVE STYLE PACK")); + assert!( + prompt.find("ARRANGEMENT CANON").unwrap() < prompt.find("ACTIVE STYLE PACK").unwrap(), + "the global canon must remain a separate authority above the selected style" + ); + std::fs::remove_dir_all(root).unwrap(); + } + #[test] fn render_config_is_injected_with_mapped_instruments() { let root = temp_project("render-config"); @@ -962,28 +1669,37 @@ mod tests { "name: forest\nsources:\n birds: birds.wav\n river: river.wav\n", ) .unwrap(); + std::fs::write( + root.join("hybrid.yaml"), + "schema_version: 1\nname: hybrid-cinematic\ndefault_palette: default\npalettes:\n default: { profile: profiles/open.yaml }\n", + ) + .unwrap(); std::fs::write( root.join("bench.json"), - r#"{"render":{"renderer":"sfizz","profile":"profiles/open.yaml","texture_profile":"profiles/forest-textures.yaml"}}"#, + r#"{"render":{"renderer":"sfizz","orchestration":"hybrid.yaml","texture_profile":"profiles/forest-textures.yaml"}}"#, ) .unwrap(); let section = render_config_section(&root); assert!(section.contains("ACTIVE RENDER CONFIGURATION")); assert!(section.contains("renderer: sfizz")); + assert!(section.contains("orchestration: hybrid.yaml (hybrid-cinematic)")); + assert!(section.contains("default palette: default")); + assert!(section.contains("palette `default`")); assert!(section.contains("piano, strings")); - assert!(section.contains("FAIL the sfizz build")); + assert!(section.contains("stable `id`")); assert!(section.contains("birds, river")); - assert!(section.contains("portable source keys")); + assert!(section.contains("inspect_textures")); + assert!(section.contains("playback modes")); let prompt = system_prompt(&root, "main", None).unwrap(); assert!(prompt.contains("ACTIVE RENDER CONFIGURATION")); std::fs::write( root.join("bench.json"), - r#"{"render":{"renderer":"sfizz","profile":"profiles/missing.yaml"}}"#, + r#"{"render":{"renderer":"sfizz","orchestration":"missing.yaml"}}"#, ) .unwrap(); let section = render_config_section(&root); - assert!(section.contains("WARNING: the profile could not be read")); + assert!(section.contains("WARNING: the orchestration could not be read")); std::fs::write( root.join("bench.json"), @@ -992,7 +1708,7 @@ mod tests { .unwrap(); let section = render_config_section(&root); assert!(section.contains("renderer: fluidsynth")); - assert!(!section.contains("mapped by this profile")); + assert!(!section.contains("mapped: ")); std::fs::remove_dir_all(root).unwrap(); } } diff --git a/src-tauri/src/agent/tools.rs b/src-tauri/src/agent/tools.rs index 17496de..15c5bd9 100644 --- a/src-tauri/src/agent/tools.rs +++ b/src-tauri/src/agent/tools.rs @@ -11,11 +11,34 @@ pub struct ToolBelt { root: PathBuf, } -#[derive(Debug)] +#[derive(Debug, Clone)] pub struct ToolResult { pub output: String, pub summary: String, pub detail: Option, + pub scene_gates: Vec, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum SceneGateKind { + /// Schema validation plus compatibility with the active bench.json render + /// configuration. + Readiness, + /// A requested build must be retried successfully; validation alone cannot + /// erase a renderer/export failure. + Build, + /// ScoreKit instrument resolution, including exact-only world identities. + InstrumentResolution, + /// One concrete scene/grammar pair. + Grammar { grammar: String }, +} + +#[derive(Debug, Clone)] +pub struct SceneGateUpdate { + pub kind: SceneGateKind, + pub path: String, + pub ready: bool, + pub reason: String, } impl ToolBelt { @@ -31,6 +54,61 @@ impl ToolBelt { .await .map_err(BenchError::io)? } + + pub fn failure_gates(&self, call: &FunctionCall, error: &BenchError) -> Vec { + failure_gates(call, error) + .into_iter() + .map(|mut update| { + if let Ok(path) = scene(&self.root, &update.path) { + if let Ok(relative) = scene_gate_path(&self.root, &path) { + update.path = relative; + } + } + update + }) + .collect() + } +} + +/// Convert a failed scene-scoped tool call into a durable request-local gate. +/// The tool error remains the authoritative payload; this only prevents a +/// later text-only response from pretending the failed check never happened. +pub fn failure_gates(call: &FunctionCall, error: &BenchError) -> Vec { + let reason = format!("{} failed: {error}", call.name); + let update = match call.name.as_str() { + "write_scene" => serde_json::from_str::(&call.arguments) + .ok() + .map(|args| (SceneGateKind::Readiness, args.path)), + "validate_scene" => serde_json::from_str::(&call.arguments) + .ok() + .map(|args| (SceneGateKind::Readiness, args.path)), + "build_scene" => serde_json::from_str::(&call.arguments) + .ok() + .map(|args| (SceneGateKind::Build, args.path)), + "inspect_instruments" => serde_json::from_str::(&call.arguments) + .ok() + .map(|args| (SceneGateKind::InstrumentResolution, args.path)), + "lint_scene" => serde_json::from_str::(&call.arguments) + .ok() + .map(|args| { + ( + SceneGateKind::Grammar { + grammar: args.grammar, + }, + args.path, + ) + }), + _ => None, + }; + update + .map(|(kind, path)| SceneGateUpdate { + kind, + path, + ready: false, + reason, + }) + .into_iter() + .collect() } pub fn definitions() -> Vec { @@ -42,8 +120,8 @@ pub fn definitions() -> Vec { ), function( "write_scene", - "Atomically write one scene YAML file inside the project. Runs `scorekit validate` plus renderer/texture-profile compatibility checks afterwards and reports the result inline; pass validate:false only when writing non-scene YAML (grammar, renderer profile, or texture profile files).", - json!({"type":"object","properties":{"path":{"type":"string"},"content":{"type":"string"},"validate":{"type":"boolean","description":"Validate the written file as a scene (default true)."}},"required":["path","content"]}), + "Atomically write one scene YAML file inside the project. Validation and active orchestration/texture-profile compatibility checks are mandatory. If any check fails, repair the scene with another write_scene call before finishing.", + json!({"type":"object","properties":{"path":{"type":"string"},"content":{"type":"string"}},"required":["path","content"]}), ), function( "validate_scene", @@ -64,12 +142,12 @@ pub fn definitions() -> Vec { "path":{"type":"string"}, "format":{"type":"string","enum":["ogg","wav"]}, "renderer":{"type":"string"}, - "sample_rate":{"type":"integer","minimum":8000,"maximum":192000}, + "sample_rate":{"type":"integer","minimum":8000,"maximum":384000}, "gain":{"type":"number","minimum":0}, "quality":{"type":"integer","minimum":0,"maximum":10}, "stems":{"type":"boolean"}, "soundfont":{"type":"string"}, - "profile":{"type":"string"}, + "orchestration":{"type":"string","description":"Project-relative scorekit orchestration profile path (--renderer sfizz only). Omit to inherit bench.json."}, "texture_profile":{"type":"string","description":"Project-relative scorekit texture profile path. Omit to inherit bench.json."} }, "required":["path"] @@ -80,6 +158,49 @@ pub fn definitions() -> Vec { "Return scorekit's semantic JSON diff for two project scenes.", json!({"type":"object","properties":{"old":{"type":"string"},"new":{"type":"string"}},"required":["old","new"]}), ), + function( + "inspect_instruments", + "Resolve every scene track through scorekit before building. Use this for palette selection and all world instruments; some schema-valid instruments require an exact external profile and cannot fall back to General MIDI.", + json!({ + "type":"object", + "properties":{ + "path":{"type":"string"}, + "orchestration":{"type":"string","description":"Project-relative orchestration profile. Omit to inherit bench.json."}, + "resolver":{"type":"string","description":"Optional project-relative scorekit resolver policy."}, + "fallback_mode":{"type":"string","enum":["strict","conservative","flexible"]}, + "verbose":{"type":"boolean"} + }, + "required":["path"] + }), + ), + function( + "inspect_textures", + "Query scorekit's structured texture catalog with exact conjunctive filters. A `no_match` result is authoritative: change the musical plan or filters, never invent a source key.", + json!({ + "type":"object", + "properties":{ + "profile":{"type":"string","description":"Project-relative texture profile. Omit to inherit bench.json."}, + "source":{"type":"string"}, + "category":{"type":"string","enum":["ambience","foley","impact","transition","tonal","industrial","organic","sound_design"]}, + "tags":{"type":"array","items":{"type":"string"}}, + "mode":{"type":"string","enum":["loop","one_shot"]}, + "use_case":{"type":"string"} + }, + "required":[] + }), + ), + function( + "check_texture_profile", + "Ask scorekit to decode and certify every structured texture source. Use before relying on a new or changed profile.", + json!({ + "type":"object", + "properties":{ + "profile":{"type":"string","description":"Project-relative texture profile. Omit to inherit bench.json."}, + "sample_rate":{"type":"integer","minimum":8000,"maximum":384000} + }, + "required":[] + }), + ), function( "doctor", "Return scorekit doctor --json verbatim.", @@ -162,27 +283,44 @@ fn execute_sync(root: &Path, call: &FunctionCall) -> Result { let args: LintArgs = args(call)?; let path = scene(root, &args.path)?; let grammar = project::resolve_inside(root, &args.grammar)?; scorekit::lint(&path, &grammar)?; - success(json!({"ok":true,"path":args.path}), "scene passes grammar") + success_with_gate( + json!({"ok":true,"path":args.path}), + "scene passes grammar", + SceneGateKind::Grammar { + grammar: args.grammar, + }, + scene_gate_path(root, &path)?, + true, + ) } "build_scene" => { let args: BuildArgs = args(call)?; @@ -202,18 +340,21 @@ fn execute_sync(root: &Path, call: &FunctionCall) -> Result Some(explicit), - // The project profile may live outside the root (GUI allows - // it), so resolve like the GUI render path does. + // The project orchestration may live outside the root (GUI + // allows it), so resolve like the GUI render path does. (None, Some(inherited)) => Some( - manifest::resolve_profile_path(root, &inherited) + manifest::resolve_config_path(root, &inherited) .to_string_lossy() .into_owned(), ), @@ -225,7 +366,7 @@ fn execute_sync(root: &Path, call: &FunctionCall) -> Result Some(explicit), (None, Some(inherited)) => Some( - manifest::resolve_profile_path(root, &inherited) + manifest::resolve_config_path(root, &inherited) .to_string_lossy() .into_owned(), ), @@ -238,22 +379,42 @@ fn execute_sync(root: &Path, call: &FunctionCall) -> Result { let args: DiffArgs = args(call)?; @@ -262,6 +423,60 @@ fn execute_sync(root: &Path, call: &FunctionCall) -> Result { + let args: InspectInstrumentsArgs = args(call)?; + let path = scene(root, &args.path)?; + let render = manifest::load(root).0.render.unwrap_or_default(); + let orchestration = resolve_explicit_or_config( + root, + args.orchestration, + render.orchestration.as_deref(), + )?; + let resolver = resolve_optional_path(root, args.resolver)?; + let report = scorekit::inspect_instruments( + &path, + orchestration.as_deref(), + resolver.as_deref(), + args.fallback_mode.as_deref(), + args.verbose.unwrap_or(false), + )?; + success_with_gate( + json!({"ok":true,"path":args.path,"report":report}), + "instrument resolution inspected", + SceneGateKind::InstrumentResolution, + scene_gate_path(root, &path)?, + true, + ) + } + "inspect_textures" => { + let args: InspectTexturesArgs = args(call)?; + let profile = active_texture_profile(root, args.profile)?; + let params = scorekit::TextureInspectParams { + source: args.source, + category: args.category, + tags: args.tags.unwrap_or_default(), + mode: args.mode, + use_case: args.use_case, + }; + let report = scorekit::inspect_textures(&profile, ¶ms)?; + let status = report + .get("status") + .and_then(Value::as_str) + .unwrap_or("unknown"); + success( + json!({"ok":true,"profile":profile,"report":report}), + format!("texture inspection: {status}"), + ) + } + "check_texture_profile" => { + let args: CheckTextureProfileArgs = args(call)?; + let profile = active_texture_profile(root, args.profile)?; + let report = scorekit::check_texture_profile(&profile, args.sample_rate)?; + success( + json!({"ok":true,"profile":profile,"report":report}), + "texture profile certified", + ) + } "doctor" => success( json!({"ok":true,"report":scorekit::doctor()?}), "doctor report ready", @@ -291,6 +506,28 @@ fn success(output: Value, summary: impl Into) -> Result, + kind: SceneGateKind, + path: String, + ready: bool, +) -> Result { + let summary = summary.into(); + Ok(ToolResult { + output: output.to_string(), + summary: summary.clone(), + detail: None, + scene_gates: vec![SceneGateUpdate { + kind, + path, + ready, + reason: summary, + }], }) } @@ -337,7 +574,6 @@ fn write_scene(root: &Path, args: WriteArgs) -> Result { (None, false) => Some(warnings.join("\n")), (None, true) => None, }; - let mut summary = String::from("scene written atomically"); let mut output = json!({ "ok": true, "path": args.path, @@ -345,52 +581,60 @@ fn write_scene(root: &Path, args: WriteArgs) -> Result { "diff": diff, "warnings": warnings }); - if args.validate.unwrap_or(true) { - let validation = match scorekit::validate(&target) { - Ok(()) => { - summary = "scene written and validated".into(); - json!({"status": "valid"}) - } - Err(error @ BenchError::Scorekit { .. }) => { - summary = format!("scene written but INVALID: {error}"); - json!({ - "status": "invalid", - "error": error, - "hint": "fix the scene with write_scene until validation passes" - }) - } - // scorekit missing or not runnable: the write itself stands. - Err(error) => { - summary = "scene written (validation unavailable)".into(); - json!({"status": "unavailable", "message": error.to_string()}) - } - }; - output["validation"] = validation; - if let Some(compat) = profile_check(root, &target) { - if !compat.is_compatible() { - summary = format!("{summary}; {}", compat.message()); - } - output["render_profile"] = serde_json::to_value(&compat).map_err(BenchError::io)?; + let (mut summary, mut ready, validation) = match scorekit::validate(&target) { + Ok(()) => ( + String::from("scene written and validated"), + true, + json!({"status": "valid"}), + ), + Err(error @ BenchError::Scorekit { .. }) => ( + format!("scene written but INVALID: {error}"), + false, + json!({ + "status": "invalid", + "error": error, + "hint": "fix the scene with write_scene until validation passes" + }), + ), + // scorekit missing or not runnable: the write itself stands, but the + // agent may not claim completion without an authoritative result. + Err(error) => ( + String::from("scene written (validation unavailable)"), + false, + json!({"status": "unavailable", "message": error.to_string()}), + ), + }; + output["validation"] = validation; + if let Some(compat) = profile_check(root, &target) { + if !compat.is_compatible() { + ready = false; + summary = format!("{summary}; {}", compat.message()); } - if let Some(compat) = texture_profile_check(root, &target) { - if !compat.is_compatible() { - summary = format!("{summary}; {}", compat.message()); - } - output["texture_profile"] = serde_json::to_value(&compat).map_err(BenchError::io)?; + output["orchestration"] = serde_json::to_value(&compat).map_err(BenchError::io)?; + } + if let Some(compat) = texture_profile_check(root, &target) { + if !compat.is_compatible() { + ready = false; + summary = format!("{summary}; {}", compat.message()); } - } else { - output["validation"] = json!({"status": "skipped"}); + output["texture_profile"] = serde_json::to_value(&compat).map_err(BenchError::io)?; } Ok(ToolResult { output: output.to_string(), - summary, + summary: summary.clone(), detail, + scene_gates: vec![SceneGateUpdate { + kind: SceneGateKind::Readiness, + path: scene_gate_path(root, &target)?, + ready, + reason: summary, + }], }) } -/// Compatibility of one scene against the project's persisted render -/// configuration (bench.json). `None` when no sfizz profile is active. -fn profile_check(root: &Path, scene_path: &Path) -> Option { +/// Compatibility of one scene against the project's persisted orchestration +/// configuration (bench.json). `None` when no sfizz orchestration is active. +fn profile_check(root: &Path, scene_path: &Path) -> Option { let render = manifest::load(root).0.render?; manifest::check_scene_profile(root, scene_path, &render) } @@ -400,19 +644,62 @@ fn texture_profile_check(root: &Path, scene_path: &Path) -> Option Result { + let mut ready = true; + if let Some(compat) = profile_check(root, scene_path) { + if !compat.is_compatible() { + ready = false; + *summary = format!("{summary}; active {}", compat.message()); + } + output["active_orchestration_compat"] = + serde_json::to_value(&compat).map_err(BenchError::io)?; + } + if let Some(compat) = texture_profile_check(root, scene_path) { + if !compat.is_compatible() { + ready = false; + *summary = format!("{summary}; active {}", compat.message()); + } + output["active_texture_profile_compat"] = + serde_json::to_value(&compat).map_err(BenchError::io)?; + } + Ok(ready) +} + +fn scene_gate_path(root: &Path, scene_path: &Path) -> Result { + let root = root.canonicalize().map_err(BenchError::io)?; + let scene = scene_path.canonicalize().map_err(BenchError::io)?; + let relative = scene.strip_prefix(&root).map_err(|_| { + BenchError::invalid(format!( + "`{}` is outside project `{}`", + scene.display(), + root.display() + )) + })?; + Ok(relative.to_string_lossy().into_owned()) +} + /// Fill build parameters the model omitted from the project render config. -/// Returns inherited renderer and texture profile paths. Pure for testing. +/// Returns inherited orchestration and texture profile paths. Pure for testing. fn inherit_render_config( renderer: &mut Option, - explicit_profile: &Option, + explicit_orchestration: &Option, explicit_texture_profile: &Option, project_render: &manifest::RenderConfig, ) -> (Option, Option) { if renderer.is_none() { renderer.clone_from(&project_render.renderer); } - let profile = if explicit_profile.is_none() && renderer.as_deref() == Some("sfizz") { - project_render.profile.clone() + let orchestration = if explicit_orchestration.is_none() && renderer.as_deref() == Some("sfizz") + { + project_render.orchestration.clone() } else { None }; @@ -421,7 +708,7 @@ fn inherit_render_config( } else { None }; - (profile, texture_profile) + (orchestration, texture_profile) } fn require_scene_path(path: &str) -> Result<(), BenchError> { @@ -447,6 +734,42 @@ fn resolve_optional(root: &Path, rel: Option) -> Result, .transpose() } +fn resolve_optional_path(root: &Path, rel: Option) -> Result, BenchError> { + rel.map(|value| project::resolve_inside(root, &value)) + .transpose() +} + +fn resolve_explicit_or_config( + root: &Path, + explicit: Option, + configured: Option<&str>, +) -> Result, BenchError> { + match explicit { + Some(path) => project::resolve_inside(root, &path).map(Some), + None => Ok(configured + .filter(|path| !path.trim().is_empty()) + .map(|path| manifest::resolve_config_path(root, path))), + } +} + +fn active_texture_profile(root: &Path, explicit: Option) -> Result { + if let Some(path) = explicit { + return project::resolve_inside(root, &path); + } + let configured = manifest::load(root) + .0 + .render + .and_then(|render| render.texture_profile) + .filter(|path| !path.trim().is_empty()) + .ok_or_else(|| { + BenchError::agent( + "invalid_tool_args", + "no texture profile supplied and bench.json has no active texture_profile", + ) + })?; + Ok(manifest::resolve_config_path(root, &configured)) +} + #[derive(Deserialize)] #[serde(deny_unknown_fields)] struct PathArgs { @@ -458,8 +781,6 @@ struct PathArgs { struct WriteArgs { path: String, content: String, - #[serde(default)] - validate: Option, } #[derive(Deserialize)] @@ -487,10 +808,38 @@ struct BuildArgs { quality: Option, stems: Option, soundfont: Option, - profile: Option, + orchestration: Option, texture_profile: Option, } +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct InspectInstrumentsArgs { + path: String, + orchestration: Option, + resolver: Option, + fallback_mode: Option, + verbose: Option, +} + +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct InspectTexturesArgs { + profile: Option, + source: Option, + category: Option, + tags: Option>, + mode: Option, + use_case: Option, +} + +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct CheckTextureProfileArgs { + profile: Option, + sample_rate: Option, +} + #[cfg(test)] mod tests { use std::sync::atomic::{AtomicU64, Ordering}; @@ -513,6 +862,90 @@ mod tests { root } + #[test] + #[cfg(unix)] + fn scene_gate_paths_do_not_collapse_backslash_filenames() { + let root = temp_project(); + std::fs::create_dir_all(root.join("dir")).unwrap(); + let backslash = root.join(r"dir\scene.yaml"); + let nested = root.join("dir/scene.yaml"); + std::fs::write(&backslash, "tracks: []\n").unwrap(); + std::fs::write(&nested, "tracks: []\n").unwrap(); + + assert_ne!( + scene_gate_path(&root, &backslash).unwrap(), + scene_gate_path(&root, &nested).unwrap() + ); + std::fs::remove_dir_all(root).unwrap(); + } + + #[test] + #[cfg(unix)] + fn failed_tool_gate_uses_the_canonical_project_relative_scene_identity() { + use std::os::unix::fs::symlink; + + let root = temp_project(); + std::fs::write(root.join("scene.yaml"), "tracks: []\n").unwrap(); + symlink("scene.yaml", root.join("alias.yaml")).unwrap(); + let belt = ToolBelt::new(root.clone()).unwrap(); + let call = FunctionCall { + id: None, + call_id: "validate".into(), + name: "validate_scene".into(), + arguments: r#"{"path":"alias.yaml"}"#.into(), + }; + let gates = belt.failure_gates(&call, &BenchError::agent("fixture", "invalid")); + + assert_eq!(gates.len(), 1); + assert_eq!(gates[0].path, "scene.yaml"); + std::fs::remove_dir_all(root).unwrap(); + } + + fn require_scorekit_06_contract() -> bool { + let handshake = scorekit::handshake(); + if handshake.compatible == Some(true) { + return true; + } + if std::env::var("SCOREBENCH_REQUIRE_SCOREKIT_CONTRACT").as_deref() == Ok("1") { + panic!( + "ScoreKit contract test was required, but the active CLI is not compatible: {:?}", + handshake.warning + ); + } + false + } + + fn write_tiny_wav(path: &Path) { + let sample_rate = 8_000_u32; + let samples = (0..800_i16) + .map(|index| { + if index % 20 < 10 { + 2_000_i16 + } else { + -2_000_i16 + } + }) + .collect::>(); + let data_size = (samples.len() * 2) as u32; + let mut bytes = Vec::with_capacity(44 + data_size as usize); + bytes.extend_from_slice(b"RIFF"); + bytes.extend_from_slice(&(36 + data_size).to_le_bytes()); + bytes.extend_from_slice(b"WAVEfmt "); + bytes.extend_from_slice(&16_u32.to_le_bytes()); + bytes.extend_from_slice(&1_u16.to_le_bytes()); + bytes.extend_from_slice(&1_u16.to_le_bytes()); + bytes.extend_from_slice(&sample_rate.to_le_bytes()); + bytes.extend_from_slice(&(sample_rate * 2).to_le_bytes()); + bytes.extend_from_slice(&2_u16.to_le_bytes()); + bytes.extend_from_slice(&16_u16.to_le_bytes()); + bytes.extend_from_slice(b"data"); + bytes.extend_from_slice(&data_size.to_le_bytes()); + for sample in samples { + bytes.extend_from_slice(&sample.to_le_bytes()); + } + std::fs::write(path, bytes).unwrap(); + } + #[tokio::test] async fn malformed_arguments_are_typed_errors() { let root = temp_project(); @@ -633,10 +1066,10 @@ mod tests { } #[tokio::test] - async fn write_scene_validate_false_skips_validation() { + async fn write_scene_rejects_the_removed_validation_bypass() { let root = temp_project(); let belt = ToolBelt::new(root.clone()).unwrap(); - let result = belt + let error = belt .execute(FunctionCall { id: None, call_id: "call".into(), @@ -649,15 +1082,17 @@ mod tests { .to_string(), }) .await - .unwrap(); - let output: serde_json::Value = serde_json::from_str(&result.output).unwrap(); - assert_eq!(output["validation"]["status"], "skipped"); - assert!(output.get("render_profile").is_none()); + .unwrap_err(); + assert!(matches!(error, BenchError::Agent { ref code, .. } if code == "invalid_tool_args")); + assert!( + !root.join("profiles/open.yaml").exists(), + "rejected arguments must not write a non-scene YAML file" + ); std::fs::remove_dir_all(root).unwrap(); } #[tokio::test] - async fn write_scene_flags_instruments_unmapped_by_active_profile() { + async fn write_scene_flags_instruments_unmapped_by_active_orchestration() { let root = temp_project(); std::fs::create_dir_all(root.join("profiles")).unwrap(); std::fs::write( @@ -665,9 +1100,14 @@ mod tests { "name: scoredata-open\ninstruments:\n piano:\n sustain: piano.sfz\n", ) .unwrap(); + std::fs::write( + root.join("hybrid.yaml"), + "schema_version: 1\nname: hybrid-cinematic\ndefault_palette: default\npalettes:\n default: { profile: profiles/open.yaml }\n", + ) + .unwrap(); std::fs::write( root.join(manifest::MANIFEST_FILE), - r#"{"render":{"renderer":"sfizz","profile":"profiles/open.yaml"}}"#, + r#"{"render":{"renderer":"sfizz","orchestration":"hybrid.yaml"}}"#, ) .unwrap(); let belt = ToolBelt::new(root.clone()).unwrap(); @@ -678,15 +1118,231 @@ mod tests { name: "write_scene".into(), arguments: serde_json::json!({ "path": "scene.yaml", - "content": "title: Hymn\ntracks:\n - instrument: choir\n pattern: pad\n" + "content": "title: Hymn\ntracks:\n - { id: choir_pad, instrument: choir, pattern: pad }\n" }) .to_string(), }) .await .unwrap(); let output: serde_json::Value = serde_json::from_str(&result.output).unwrap(); - assert_eq!(output["render_profile"]["unmapped"][0], "choir"); - assert!(result.summary.contains("`choir`"), "{}", result.summary); + assert_eq!( + output["orchestration"]["tracks"][0]["track_id"], + "choir_pad" + ); + assert!(output["orchestration"]["tracks"][0]["error"] + .as_str() + .unwrap() + .contains("choir")); + assert!(result.summary.contains("`choir_pad`"), "{}", result.summary); + std::fs::remove_dir_all(root).unwrap(); + } + + #[test] + fn failed_scene_tools_create_independent_retry_gates() { + let error = BenchError::agent("fixture", "failed"); + let cases = [ + ( + FunctionCall { + id: None, + call_id: "validate".into(), + name: "validate_scene".into(), + arguments: r#"{"path":"scene.yaml"}"#.into(), + }, + SceneGateKind::Readiness, + ), + ( + FunctionCall { + id: None, + call_id: "build".into(), + name: "build_scene".into(), + arguments: r#"{"path":"scene.yaml"}"#.into(), + }, + SceneGateKind::Build, + ), + ( + FunctionCall { + id: None, + call_id: "instruments".into(), + name: "inspect_instruments".into(), + arguments: r#"{"path":"scene.yaml"}"#.into(), + }, + SceneGateKind::InstrumentResolution, + ), + ]; + + for (call, expected) in cases { + let updates = failure_gates(&call, &error); + assert_eq!(updates.len(), 1); + assert_eq!(updates[0].kind, expected); + assert_eq!(updates[0].path, "scene.yaml"); + assert!(!updates[0].ready); + } + } + + #[test] + fn explicit_build_override_cannot_clear_incompatible_active_texture_profile() { + let root = temp_project(); + std::fs::create_dir_all(root.join("profiles")).unwrap(); + std::fs::write( + root.join("scene.yaml"), + "textures:\n - { source: birds, mode: loop }\n", + ) + .unwrap(); + std::fs::write( + root.join("profiles/active.yaml"), + "schema_version: 1\nname: active\nsources:\n birds:\n path: birds.wav\n description: One bird call\n category: organic\n tags: [bird]\n playback:\n modes: [one_shot]\n default_mode: one_shot\n use_cases: [forest]\n provenance:\n library: fixture@1.0.0\n", + ) + .unwrap(); + std::fs::write( + root.join(manifest::MANIFEST_FILE), + r#"{"render":{"texture_profile":"profiles/active.yaml"}}"#, + ) + .unwrap(); + + let mut output = json!({"ok": true, "texture_profile": "profiles/override.yaml"}); + let mut summary = "built out/scene.ogg".to_owned(); + let ready = + append_active_compatibility(&root, &root.join("scene.yaml"), &mut output, &mut summary) + .unwrap(); + + assert!(!ready); + assert_eq!( + output["active_texture_profile_compat"]["mode_mismatches"][0]["requested"], + "loop" + ); + assert!(summary.contains("active")); + assert!(summary.contains("one_shot")); + std::fs::remove_dir_all(root).unwrap(); + } + + #[tokio::test] + async fn scorekit_06_contract_tools_query_live_capabilities() { + if !require_scorekit_06_contract() { + return; + } + let root = temp_project(); + std::fs::create_dir_all(root.join("profiles")).unwrap(); + std::fs::write( + root.join("forest.yaml"), + include_str!("../../tests/fixtures/scenes/forest.yaml"), + ) + .unwrap(); + write_tiny_wav(&root.join("profiles/tone.wav")); + std::fs::write( + root.join("profiles/textures.yaml"), + "schema_version: 1\nname: contract-textures\nsources:\n tone:\n path: tone.wav\n description: Deterministic contract tone\n category: tonal\n tags: [tone, test]\n playback:\n modes: [loop]\n default_mode: loop\n use_cases: [test]\n provenance:\n library: scorebench-contract@1.0.0\n", + ) + .unwrap(); + let belt = ToolBelt::new(root.clone()).unwrap(); + + let instruments = belt + .execute(FunctionCall { + id: None, + call_id: "instruments".into(), + name: "inspect_instruments".into(), + arguments: serde_json::json!({ + "path":"forest.yaml", + "orchestration":null, + "resolver":null, + "fallback_mode":"conservative", + "verbose":false + }) + .to_string(), + }) + .await + .unwrap(); + let instruments: Value = serde_json::from_str(&instruments.output).unwrap(); + assert_eq!(instruments["report"]["summary"]["missing"], 0); + assert_eq!(instruments["report"]["tracks"][0]["status"], "exact"); + + std::fs::write( + root.join("world.yaml"), + "tempo: 80\nbars: 2\ntracks:\n - { id: lead, instrument: erhu, pattern: sustain }\n", + ) + .unwrap(); + let world_error = belt + .execute(FunctionCall { + id: None, + call_id: "world".into(), + name: "inspect_instruments".into(), + arguments: serde_json::json!({ + "path":"world.yaml", + "orchestration":null, + "resolver":null, + "fallback_mode":"flexible", + "verbose":false + }) + .to_string(), + }) + .await + .unwrap_err(); + let BenchError::Scorekit { code, report, .. } = world_error else { + panic!("world-instrument resolution must be a structured scorekit error"); + }; + assert_eq!(code, "resolution"); + assert_eq!( + report.unwrap()["tracks"][0]["best_candidate"]["rejected"], + "world_instrument_requires_exact_source" + ); + + let textures = belt + .execute(FunctionCall { + id: None, + call_id: "textures".into(), + name: "inspect_textures".into(), + arguments: serde_json::json!({ + "profile":"profiles/textures.yaml", + "source":null, + "category":"tonal", + "tags":["tone"], + "mode":"loop", + "use_case":"test" + }) + .to_string(), + }) + .await + .unwrap(); + let textures: Value = serde_json::from_str(&textures.output).unwrap(); + assert_eq!(textures["report"]["status"], "match"); + assert_eq!(textures["report"]["sources"][0]["source"], "tone"); + + let no_match = belt + .execute(FunctionCall { + id: None, + call_id: "no-match".into(), + name: "inspect_textures".into(), + arguments: serde_json::json!({ + "profile":"profiles/textures.yaml", + "source":null, + "category":null, + "tags":["absent"], + "mode":null, + "use_case":null + }) + .to_string(), + }) + .await + .unwrap(); + let no_match: Value = serde_json::from_str(&no_match.output).unwrap(); + assert_eq!(no_match["report"]["status"], "no_match"); + assert_eq!(no_match["report"]["matched"], 0); + + let check = belt + .execute(FunctionCall { + id: None, + call_id: "check".into(), + name: "check_texture_profile".into(), + arguments: serde_json::json!({ + "profile":"profiles/textures.yaml", + "sample_rate":8000 + }) + .to_string(), + }) + .await + .unwrap(); + let check: Value = serde_json::from_str(&check.output).unwrap(); + assert_eq!(check["report"]["passed"], 1); + assert_eq!(check["report"]["failed"], 0); std::fs::remove_dir_all(root).unwrap(); } @@ -694,7 +1350,7 @@ mod tests { fn build_inherits_project_render_config_unless_overridden() { let project_render = manifest::RenderConfig { renderer: Some("sfizz".into()), - profile: Some("profiles/open.yaml".into()), + orchestration: Some("hybrid.yaml".into()), texture_profile: Some("profiles/forest-textures.yaml".into()), }; @@ -702,7 +1358,7 @@ mod tests { let (inherited, texture) = inherit_render_config(&mut renderer, &None, &None, &project_render); assert_eq!(renderer.as_deref(), Some("sfizz")); - assert_eq!(inherited.as_deref(), Some("profiles/open.yaml")); + assert_eq!(inherited.as_deref(), Some("hybrid.yaml")); assert_eq!(texture.as_deref(), Some("profiles/forest-textures.yaml")); // Explicit renderer wins; texture profiles are renderer-independent. @@ -713,7 +1369,7 @@ mod tests { assert!(inherited.is_none()); assert_eq!(texture.as_deref(), Some("profiles/forest-textures.yaml")); - // Explicit profile wins over the project profile. + // Explicit orchestration wins over the project orchestration. let mut renderer = None; let explicit = Some("other.yaml".to_owned()); let (inherited, texture) = @@ -743,7 +1399,14 @@ mod tests { .collect::>(); names.sort_unstable(); names.dedup(); - assert_eq!(names.len(), 8); + assert_eq!(names.len(), 11); + for expected in [ + "inspect_instruments", + "inspect_textures", + "check_texture_profile", + ] { + assert!(names.contains(&expected), "missing tool `{expected}`"); + } } #[test] @@ -781,14 +1444,31 @@ mod tests { build.parameters["properties"]["texture_profile"]["type"], serde_json::json!(["string", "null"]) ); + assert_eq!( + build.parameters["properties"]["orchestration"]["type"], + serde_json::json!(["string", "null"]) + ); + assert_eq!( + build.parameters["properties"]["sample_rate"]["maximum"], + serde_json::json!(384_000) + ); let write = definitions() .into_iter() .find(|definition| definition.name == "write_scene") .unwrap(); + assert!( + write.parameters["properties"].get("validate").is_none(), + "scene validation must not be model-optional" + ); + + let texture_check = definitions() + .into_iter() + .find(|definition| definition.name == "check_texture_profile") + .unwrap(); assert_eq!( - write.parameters["properties"]["validate"]["type"], - serde_json::json!(["boolean", "null"]) + texture_check.parameters["properties"]["sample_rate"]["maximum"], + serde_json::json!(384_000) ); } } diff --git a/src-tauri/src/arrangement.rs b/src-tauri/src/arrangement.rs new file mode 100644 index 0000000..de91be1 --- /dev/null +++ b/src-tauri/src/arrangement.rs @@ -0,0 +1,215 @@ +//! Versioned, built-in arrangement doctrine for the composing agent. + +use std::sync::OnceLock; + +use serde::Deserialize; + +const CANON_SOURCE: &str = include_str!("../arrangement/canon.yaml"); + +#[derive(Debug)] +pub struct ArrangementCanon { + pub schema_version: u32, + pub id: String, + pub version: String, + pub source_scorekit_range: String, + pub source_commit: String, + source: &'static str, +} + +#[derive(Deserialize)] +struct Envelope { + schema_version: u32, + id: String, + version: String, + source_scorekit_range: String, + provenance: Provenance, +} + +#[derive(Deserialize)] +struct Provenance { + commit: String, +} + +pub fn canon() -> &'static ArrangementCanon { + static CANON: OnceLock = OnceLock::new(); + CANON.get_or_init(|| { + let envelope: Envelope = serde_yaml::from_str(CANON_SOURCE) + .expect("embedded arrangement canon must be valid YAML"); + ArrangementCanon { + schema_version: envelope.schema_version, + id: envelope.id, + version: envelope.version, + source_scorekit_range: envelope.source_scorekit_range, + source_commit: envelope.provenance.commit, + source: CANON_SOURCE, + } + }) +} + +pub fn prompt_section() -> String { + let canon = canon(); + format!( + "ARRANGEMENT CANON `{}` v{} (schema {}, source scorekit {} @ {}):\n\ + Apply this versioned doctrine to every compose or arrangement-revision turn. \ + The live ScoreKit schemas remain authoritative if this canon and the installed \ + tool disagree.\n{}\n", + canon.id, + canon.version, + canon.schema_version, + canon.source_scorekit_range, + canon.source_commit, + canon.source.trim_end() + ) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn canon_has_a_typed_versioned_scorekit_v06_envelope() { + let canon = canon(); + + assert_eq!(canon.schema_version, 1); + assert_eq!(canon.id, "scorekit-arrangement-canon"); + assert_eq!(canon.version, "1.0.0"); + assert_eq!(canon.source_scorekit_range, ">=0.6.0, <0.7.0"); + assert_eq!( + canon.source_scorekit_range, + crate::scorekit::TESTED_SCOREKIT_RANGE + ); + assert_eq!(canon.source_commit, "c2e0b0c"); + assert!(canon.source.contains("skills/scorekit/SKILL.md")); + assert!( + (4 * 1024..=7 * 1024).contains(&canon.source.len()), + "canon must stay prompt-sized, got {} bytes", + canon.source.len() + ); + } + + #[test] + fn prompt_requires_a_falsifiable_intent_contract_and_live_schema_only() { + let prompt = prompt_section(); + + for required in [ + "ARRANGEMENT CANON `scorekit-arrangement-canon` v1.0.0", + "scorekit schema --json", + "before_write", + "expected_changes", + "after_write", + "never invent a scene field, instrument, palette, articulation, or texture source", + ] { + assert!(prompt.contains(required), "missing canon rule: {required}"); + } + } + + #[test] + fn prompt_breaks_palette_inertia_before_tracks_are_written() { + let prompt = prompt_section(); + + for required in [ + "name_the_inertia_answer", + "candidate_count: \"2..=3\"", + "minimum_changed_axes: 3", + "Orchestral strings must win this comparison; they never inherit by default.", + ] { + assert!( + prompt.contains(required), + "missing palette rule: {required}" + ); + } + } + + #[test] + fn prompt_carries_the_measured_craft_contract() { + let prompt = prompt_section(); + + for required in [ + "core_identity_count: 1", + "total_motifs_max: 2", + "simultaneous_non_rest_melody_voices_max: 2", + "rest_ratio_per_melody_min: 0.30", + "long_rest_note_beats_max: 16", + "shape: \"rise -> peak -> cut_or_contrast -> return\"", + "loop_samples and total_samples", + "last harmony must pull toward the first", + ] { + assert!(prompt.contains(required), "missing craft rule: {required}"); + } + } + + #[test] + fn prompt_keeps_every_world_identity_exact_only() { + let prompt = prompt_section(); + + for identity in [ + "erhu", + "pipa", + "guzheng", + "dizi", + "shakuhachi", + "shamisen", + "sitar", + "tabla", + "oud", + "ney", + "duduk", + ] { + assert!( + prompt.contains(identity), + "missing world identity: {identity}" + ); + } + assert!(prompt.contains("fallback_policy: exact_only")); + assert!(prompt.contains("Never substitute into, out of, or within this family.")); + assert!(prompt.contains("gm_exact_melodic: [shakuhachi, shamisen, sitar]")); + assert!(prompt.contains("tabla requires pattern tabla")); + } + + #[test] + fn prompt_discovers_and_certifies_textures_instead_of_guessing() { + let prompt = prompt_section(); + + for required in [ + "Never guess textures[].source", + "scorekit --json texture inspect ", + "scorekit --json texture inspect --source ", + "scorekit --json texture check ", + "playback.modes", + "no_match means re-orchestrate or add a real source", + ] { + assert!( + prompt.contains(required), + "missing texture rule: {required}" + ); + } + } + + #[test] + fn prompt_orders_validate_lint_inspect_texture_and_build_gates() { + let prompt = prompt_section(); + let ordered = [ + "G1_validate", + "G2_lint", + "G3_inspect_instruments", + "G4_textures", + "G5_build_evidence", + ] + .map(|gate| { + prompt + .find(gate) + .unwrap_or_else(|| panic!("missing gate: {gate}")) + }); + assert!(ordered.windows(2).all(|pair| pair[0] < pair[1])); + + for required in [ + "scorekit --json validate ", + "scorekit --json lint --grammar ", + "scorekit --json inspect-instruments [--orchestration ]", + "scorekit build -o ", + "failed_blocking_gate: do_not_report_completion", + ] { + assert!(prompt.contains(required), "missing gate rule: {required}"); + } + } +} diff --git a/src-tauri/src/bin/capability_eval.rs b/src-tauri/src/bin/capability_eval.rs new file mode 100644 index 0000000..b1d4dd2 --- /dev/null +++ b/src-tauri/src/bin/capability_eval.rs @@ -0,0 +1,137 @@ +use std::{env, path::PathBuf, process::ExitCode}; + +use scorebench_lib::capability; +use serde_json::{Map, Value}; + +const USAGE: &str = "Usage: capability_eval [--min-pass-rate <0..1>] [--pretty]"; + +struct Cli { + case_path: PathBuf, + minimum_pass_rate: Option, + pretty: bool, +} + +enum ParseResult { + Run(Cli), + Help, +} + +fn main() -> ExitCode { + match parse_args(env::args().skip(1)) { + Ok(ParseResult::Help) => { + println!("{USAGE}"); + ExitCode::SUCCESS + } + Ok(ParseResult::Run(cli)) => run(cli), + Err(error) => { + eprintln!("error: {error}"); + eprintln!("{USAGE}"); + ExitCode::from(2) + } + } +} + +fn run(cli: Cli) -> ExitCode { + let report = match capability::evaluate_case_file(&cli.case_path) { + Ok(report) => report, + Err(error) => { + eprintln!("error: {error}"); + return ExitCode::from(2); + } + }; + + let threshold_met = cli + .minimum_pass_rate + .map(|minimum| report.meets_threshold(minimum)); + let mut output = match serde_json::to_value(&report) { + Ok(Value::Object(object)) => object, + Ok(_) => { + eprintln!("error: capability report did not serialize as a JSON object"); + return ExitCode::from(2); + } + Err(error) => { + eprintln!("error: cannot serialize capability report: {error}"); + return ExitCode::from(2); + } + }; + add_threshold_fields(&mut output, cli.minimum_pass_rate, threshold_met); + + let serialized = if cli.pretty { + serde_json::to_string_pretty(&output) + } else { + serde_json::to_string(&output) + }; + match serialized { + Ok(json) => println!("{json}"), + Err(error) => { + eprintln!("error: cannot serialize capability report: {error}"); + return ExitCode::from(2); + } + } + + match threshold_met { + Some(false) => ExitCode::FAILURE, + Some(true) | None => ExitCode::SUCCESS, + } +} + +fn add_threshold_fields( + output: &mut Map, + minimum_pass_rate: Option, + threshold_met: Option, +) { + if let (Some(minimum), Some(met)) = (minimum_pass_rate, threshold_met) { + output.insert("minimum_pass_rate".to_owned(), Value::from(minimum)); + output.insert("threshold_met".to_owned(), Value::from(met)); + } +} + +fn parse_args(args: impl IntoIterator) -> Result { + let mut case_path = None; + let mut minimum_pass_rate = None; + let mut pretty = false; + let mut args = args.into_iter(); + + while let Some(argument) = args.next() { + match argument.as_str() { + "-h" | "--help" => return Ok(ParseResult::Help), + "--pretty" => pretty = true, + "--min-pass-rate" => { + let raw = args + .next() + .ok_or_else(|| "--min-pass-rate requires a value".to_owned())?; + minimum_pass_rate = Some(parse_threshold(&raw)?); + } + _ if argument.starts_with("--min-pass-rate=") => { + let raw = argument + .strip_prefix("--min-pass-rate=") + .expect("prefix was checked"); + minimum_pass_rate = Some(parse_threshold(raw)?); + } + _ if argument.starts_with('-') => { + return Err(format!("unknown option '{argument}'")); + } + _ if case_path.is_none() => case_path = Some(PathBuf::from(argument)), + _ => return Err(format!("unexpected argument '{argument}'")), + } + } + + let case_path = case_path.ok_or_else(|| "a capability case path is required".to_owned())?; + Ok(ParseResult::Run(Cli { + case_path, + minimum_pass_rate, + pretty, + })) +} + +fn parse_threshold(raw: &str) -> Result { + let value = raw + .parse::() + .map_err(|_| format!("invalid pass-rate threshold '{raw}'"))?; + if !value.is_finite() || !(0.0..=1.0).contains(&value) { + return Err(format!( + "pass-rate threshold must be a finite number between 0 and 1, got '{raw}'" + )); + } + Ok(value) +} diff --git a/src-tauri/src/capability.rs b/src-tauri/src/capability.rs new file mode 100644 index 0000000..95b3b47 --- /dev/null +++ b/src-tauri/src/capability.rs @@ -0,0 +1,575 @@ +//! Pure, offline capability evaluation for arrangement intent cases. +//! +//! A case names one `before` scene, one or more candidate `after` scenes, and +//! the musical directions that every candidate should satisfy. Evaluation only +//! reads YAML and computes structural metrics; it never invokes a model, +//! `scorekit`, or an audio renderer. + +use std::{ + collections::{BTreeMap, BTreeSet}, + error::Error, + fmt, fs, + path::{Path, PathBuf}, +}; + +use serde::{Deserialize, Serialize}; +use serde_json::{json, Value as JsonValue}; + +const SUPPORTED_SCHEMA_VERSION: u32 = 1; +const FLOAT_EPSILON: f64 = 1.0e-9; + +/// A complete capability-case evaluation. +#[derive(Debug, Serialize)] +pub struct CapabilityReport { + pub schema_version: u32, + pub case_id: String, + pub intent: String, + pub sample_count: usize, + pub passed_samples: usize, + pub pass_rate: f64, + pub samples: Vec, +} + +impl CapabilityReport { + /// Return whether the report satisfies a pass-rate threshold in `[0, 1]`. + pub fn meets_threshold(&self, minimum_pass_rate: f64) -> bool { + minimum_pass_rate.is_finite() + && (0.0..=1.0).contains(&minimum_pass_rate) + && self.pass_rate + FLOAT_EPSILON >= minimum_pass_rate + } +} + +/// Evaluation details for one candidate scene. +#[derive(Debug, Serialize)] +pub struct SampleResult { + pub id: String, + pub path: String, + pub passed: bool, + #[serde(skip_serializing_if = "Option::is_none")] + pub error: Option, + pub assertions: Vec, +} + +/// One measurable expectation evaluated against a candidate scene. +#[derive(Debug, Serialize)] +pub struct AssertionResult { + pub name: String, + pub expectation: String, + pub passed: bool, + pub before: JsonValue, + pub after: JsonValue, +} + +/// An input, schema, or baseline error that prevents case evaluation. +#[derive(Debug)] +pub struct CapabilityError { + message: String, +} + +impl CapabilityError { + fn new(message: impl Into) -> Self { + Self { + message: message.into(), + } + } +} + +impl fmt::Display for CapabilityError { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + formatter.write_str(&self.message) + } +} + +impl Error for CapabilityError {} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct CapabilityCase { + schema_version: u32, + id: String, + intent: String, + before: PathBuf, + after_samples: Vec, + assertions: Assertions, +} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct AfterSample { + id: String, + path: PathBuf, +} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct Assertions { + tempo: Direction, + track_count: Direction, + melody_rest_ratio: Direction, + mode: ModeExpectation, + r#loop: Preservation, + motif_contour: Preservation, +} + +#[derive(Clone, Copy, Debug, Deserialize)] +#[serde(rename_all = "snake_case")] +enum Direction { + Increase, + Decrease, + Preserve, +} + +impl Direction { + fn as_str(self) -> &'static str { + match self { + Self::Increase => "increase", + Self::Decrease => "decrease", + Self::Preserve => "preserve", + } + } + + fn compare_f64(self, before: f64, after: f64) -> bool { + match self { + Self::Increase => after > before + FLOAT_EPSILON, + Self::Decrease => after < before - FLOAT_EPSILON, + Self::Preserve => (after - before).abs() <= FLOAT_EPSILON, + } + } + + fn compare_usize(self, before: usize, after: usize) -> bool { + match self { + Self::Increase => after > before, + Self::Decrease => after < before, + Self::Preserve => after == before, + } + } +} + +#[derive(Clone, Copy, Debug, Deserialize)] +#[serde(rename_all = "snake_case")] +enum ModeExpectation { + ToMinor, + ToMajor, + Preserve, +} + +impl ModeExpectation { + fn as_str(self) -> &'static str { + match self { + Self::ToMinor => "to_minor", + Self::ToMajor => "to_major", + Self::Preserve => "preserve", + } + } + + fn compare(self, before: Mode, after: Mode) -> bool { + match self { + Self::ToMinor => after == Mode::Minor, + Self::ToMajor => after == Mode::Major, + Self::Preserve => after == before, + } + } +} + +#[derive(Clone, Copy, Debug, Deserialize)] +#[serde(rename_all = "snake_case")] +enum Preservation { + Preserve, +} + +impl Preservation { + fn as_str(self) -> &'static str { + match self { + Self::Preserve => "preserve", + } + } +} + +#[derive(Debug, Deserialize)] +struct Scene { + tempo: f64, + key: String, + #[serde(rename = "loop")] + loop_enabled: bool, + motifs: BTreeMap>, + tracks: Vec, +} + +#[derive(Debug, Deserialize)] +struct MotifNote { + degree: i64, + beats: f64, +} + +#[derive(Debug, Deserialize)] +struct Track { + pattern: String, + motif: Option, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize)] +#[serde(rename_all = "snake_case")] +enum Mode { + Major, + Minor, +} + +#[derive(Debug)] +struct SceneMetrics { + tempo: f64, + track_count: usize, + melody_rest_ratio: f64, + mode: Mode, + loop_enabled: bool, + motif_contours: BTreeMap>, +} + +/// Evaluate a YAML capability case and every candidate it references. +/// +/// Paths in the case are resolved relative to the case file. A malformed +/// candidate is counted as a failed sample and is represented with an `error` +/// field so one bad generation does not discard the rest of the run. +pub fn evaluate_case_file(path: &Path) -> Result { + let case: CapabilityCase = read_yaml(path, "capability case")?; + validate_case(&case)?; + + let base_dir = path.parent().unwrap_or_else(|| Path::new(".")); + let before_path = resolve_path(base_dir, &case.before); + let before = read_scene_metrics(&before_path).map_err(|error| { + CapabilityError::new(format!( + "cannot evaluate baseline '{}': {error}", + before_path.display() + )) + })?; + + if !before + .motif_contours + .values() + .any(|contour| !contour.is_empty()) + { + return Err(CapabilityError::new(format!( + "baseline '{}' has no motif with at least two pitched notes", + before_path.display() + ))); + } + + let samples = case + .after_samples + .iter() + .map(|sample| evaluate_sample(base_dir, sample, &before, &case.assertions)) + .collect::>(); + let sample_count = samples.len(); + let passed_samples = samples.iter().filter(|sample| sample.passed).count(); + let pass_rate = passed_samples as f64 / sample_count as f64; + + Ok(CapabilityReport { + schema_version: case.schema_version, + case_id: case.id, + intent: case.intent, + sample_count, + passed_samples, + pass_rate, + samples, + }) +} + +fn validate_case(case: &CapabilityCase) -> Result<(), CapabilityError> { + if case.schema_version != SUPPORTED_SCHEMA_VERSION { + return Err(CapabilityError::new(format!( + "unsupported capability case schema_version {}; expected {}", + case.schema_version, SUPPORTED_SCHEMA_VERSION + ))); + } + if case.id.trim().is_empty() { + return Err(CapabilityError::new("capability case id must not be empty")); + } + if case.intent.trim().is_empty() { + return Err(CapabilityError::new( + "capability case intent must not be empty", + )); + } + if case.after_samples.is_empty() { + return Err(CapabilityError::new( + "capability case must contain at least one after sample", + )); + } + + let mut ids = BTreeSet::new(); + for sample in &case.after_samples { + if sample.id.trim().is_empty() { + return Err(CapabilityError::new("after sample id must not be empty")); + } + if !ids.insert(sample.id.as_str()) { + return Err(CapabilityError::new(format!( + "duplicate after sample id '{}'", + sample.id + ))); + } + } + Ok(()) +} + +fn evaluate_sample( + base_dir: &Path, + sample: &AfterSample, + before: &SceneMetrics, + expected: &Assertions, +) -> SampleResult { + let path = resolve_path(base_dir, &sample.path); + let path_text = sample.path.to_string_lossy().into_owned(); + + match read_scene_metrics(&path) { + Ok(after) => { + let assertions = evaluate_assertions(before, &after, expected); + let passed = assertions.iter().all(|assertion| assertion.passed); + SampleResult { + id: sample.id.clone(), + path: path_text, + passed, + error: None, + assertions, + } + } + Err(error) => SampleResult { + id: sample.id.clone(), + path: path_text, + passed: false, + error: Some(format!("cannot evaluate '{}': {error}", path.display())), + assertions: Vec::new(), + }, + } +} + +fn evaluate_assertions( + before: &SceneMetrics, + after: &SceneMetrics, + expected: &Assertions, +) -> Vec { + vec![ + AssertionResult { + name: "tempo".to_owned(), + expectation: expected.tempo.as_str().to_owned(), + passed: expected.tempo.compare_f64(before.tempo, after.tempo), + before: json!(before.tempo), + after: json!(after.tempo), + }, + AssertionResult { + name: "track_count".to_owned(), + expectation: expected.track_count.as_str().to_owned(), + passed: expected + .track_count + .compare_usize(before.track_count, after.track_count), + before: json!(before.track_count), + after: json!(after.track_count), + }, + AssertionResult { + name: "melody_rest_ratio".to_owned(), + expectation: expected.melody_rest_ratio.as_str().to_owned(), + passed: expected + .melody_rest_ratio + .compare_f64(before.melody_rest_ratio, after.melody_rest_ratio), + before: json!(before.melody_rest_ratio), + after: json!(after.melody_rest_ratio), + }, + AssertionResult { + name: "mode".to_owned(), + expectation: expected.mode.as_str().to_owned(), + passed: expected.mode.compare(before.mode, after.mode), + before: json!(before.mode), + after: json!(after.mode), + }, + AssertionResult { + name: "loop".to_owned(), + expectation: expected.r#loop.as_str().to_owned(), + passed: before.loop_enabled == after.loop_enabled, + before: json!(before.loop_enabled), + after: json!(after.loop_enabled), + }, + AssertionResult { + name: "motif_contour".to_owned(), + expectation: expected.motif_contour.as_str().to_owned(), + passed: before.motif_contours == after.motif_contours, + before: json!(before.motif_contours), + after: json!(after.motif_contours), + }, + ] +} + +fn read_scene_metrics(path: &Path) -> Result { + let scene: Scene = read_yaml(path, "scene")?; + + if !scene.tempo.is_finite() || scene.tempo <= 0.0 { + return Err(CapabilityError::new(format!( + "scene '{}' tempo must be a positive finite number", + path.display() + ))); + } + if scene.motifs.is_empty() { + return Err(CapabilityError::new(format!( + "scene '{}' must contain at least one motif", + path.display() + ))); + } + + let mode = parse_mode(&scene.key).ok_or_else(|| { + CapabilityError::new(format!( + "scene '{}' key '{}' does not identify major or minor mode", + path.display(), + scene.key + )) + })?; + let melody_rest_ratio = melody_rest_ratio(&scene, path)?; + let motif_contours = motif_contours(&scene, path)?; + + Ok(SceneMetrics { + tempo: scene.tempo, + track_count: scene.tracks.len(), + melody_rest_ratio, + mode, + loop_enabled: scene.loop_enabled, + motif_contours, + }) +} + +fn melody_rest_ratio(scene: &Scene, path: &Path) -> Result { + let mut total_beats = 0.0; + let mut rest_beats = 0.0; + let mut melody_tracks = 0_usize; + + for track in &scene.tracks { + if !track.pattern.eq_ignore_ascii_case("melody") { + continue; + } + melody_tracks += 1; + let motif_name = track.motif.as_deref().ok_or_else(|| { + CapabilityError::new(format!( + "scene '{}' melody track is missing a motif reference", + path.display() + )) + })?; + let notes = scene.motifs.get(motif_name).ok_or_else(|| { + CapabilityError::new(format!( + "scene '{}' melody track references missing motif '{}'", + path.display(), + motif_name + )) + })?; + + for note in notes { + validate_note_duration(note, path, motif_name)?; + total_beats += note.beats; + if note.degree == 0 { + rest_beats += note.beats; + } + } + } + + if melody_tracks == 0 || total_beats <= 0.0 { + return Err(CapabilityError::new(format!( + "scene '{}' needs at least one non-empty melody track", + path.display() + ))); + } + Ok(rest_beats / total_beats) +} + +fn motif_contours( + scene: &Scene, + path: &Path, +) -> Result>, CapabilityError> { + scene + .motifs + .iter() + .map(|(name, notes)| { + for note in notes { + validate_note_duration(note, path, name)?; + } + + let pitched_degrees = notes + .iter() + .filter(|note| note.degree != 0) + .map(|note| note.degree) + .collect::>(); + let contour = pitched_degrees + .windows(2) + .map(|pair| (pair[1] - pair[0]).signum() as i8) + .collect::>(); + Ok((name.clone(), contour)) + }) + .collect() +} + +fn validate_note_duration( + note: &MotifNote, + path: &Path, + motif_name: &str, +) -> Result<(), CapabilityError> { + if !note.beats.is_finite() || note.beats <= 0.0 { + return Err(CapabilityError::new(format!( + "scene '{}' motif '{}' contains a non-positive or non-finite duration", + path.display(), + motif_name + ))); + } + Ok(()) +} + +fn parse_mode(key: &str) -> Option { + let normalized = key.trim().to_ascii_lowercase(); + if normalized.ends_with("minor") { + Some(Mode::Minor) + } else if normalized.ends_with("major") { + Some(Mode::Major) + } else { + None + } +} + +fn resolve_path(base_dir: &Path, path: &Path) -> PathBuf { + if path.is_absolute() { + path.to_owned() + } else { + base_dir.join(path) + } +} + +fn read_yaml(path: &Path, kind: &str) -> Result +where + T: for<'de> Deserialize<'de>, +{ + let contents = fs::read_to_string(path).map_err(|error| { + CapabilityError::new(format!("cannot read {kind} '{}': {error}", path.display())) + })?; + serde_yaml::from_str(&contents).map_err(|error| { + CapabilityError::new(format!("cannot parse {kind} '{}': {error}", path.display())) + }) +} + +#[cfg(test)] +mod tests { + use std::path::Path; + + use super::*; + + #[test] + fn one_passing_and_one_failing_sample_produce_half_pass_rate() { + let case = Path::new(env!("CARGO_MANIFEST_DIR")) + .join("tests/fixtures/capability/lonelier.case.yaml"); + + let report = evaluate_case_file(&case).expect("fixture case evaluates"); + + assert_eq!(report.case_id, "lonelier"); + assert_eq!( + report.intent, + "Make it lonelier without losing the motif or loop." + ); + assert_eq!(report.sample_count, 2); + assert_eq!(report.passed_samples, 1); + assert_eq!(report.pass_rate, 0.5); + assert!(report.samples[0].passed); + assert!(!report.samples[1].passed); + assert_eq!(report.samples[0].assertions.len(), 6); + assert!(report.meets_threshold(0.5)); + assert!(!report.meets_threshold(0.75)); + } +} diff --git a/src-tauri/src/error.rs b/src-tauri/src/error.rs index ec2d94b..4a55f61 100644 --- a/src-tauri/src/error.rs +++ b/src-tauri/src/error.rs @@ -14,6 +14,11 @@ pub enum BenchError { exit_code: i32, field: Option, location: Option, + /// Structured diagnosis some failures carry (for example the + /// instrument-resolution report or v0.6 texture certification report). + /// Passed through untouched, and boxed so this rare payload does not + /// widen every `Result` in the app. + report: Option>, }, /// Local I/O failure (project scan, asset read, ...). Io { message: String }, diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs index c00df44..8bea593 100644 --- a/src-tauri/src/lib.rs +++ b/src-tauri/src/lib.rs @@ -1,5 +1,7 @@ mod agent; +mod arrangement; mod attachments; +pub mod capability; mod error; mod llm; mod manifest; @@ -13,7 +15,7 @@ mod settings; mod styles; mod watcher; -use std::path::PathBuf; +use std::path::{Path, PathBuf}; use tauri::ipc::{Channel, Response}; use tauri::{AppHandle, Manager, State}; @@ -106,6 +108,29 @@ fn active_style( } } +fn persist_agent_outcome( + state: &agent::AgentState, + root: &Path, + session: &str, + outcome: &agent::RunOutcome, + mut emit: impl FnMut(agent::AgentEvent), +) -> Result { + if outcome.status != agent::RunStatus::Complete { + emit(agent::AgentEvent::TextRollback); + return Ok(false); + } + match state.complete(root, session, outcome.history.clone()) { + Ok(()) => { + emit(agent::AgentEvent::TextFinalize); + Ok(true) + } + Err(error) => { + emit(agent::AgentEvent::TextRollback); + Err(error) + } + } +} + /// Persist bytes pasted into the chat composer to a temp file so the normal /// path-based attachment pipeline can pick them up. Returns the stashed path. #[tauri::command] @@ -211,38 +236,53 @@ async fn send_chat( let prompt_tokens = outcome .prompt_tokens .unwrap_or_else(|| memory::estimate_tokens(&outcome.history)); - state.complete(&root, &session, outcome.history.clone())?; - if prompt_tokens > context_budget_tokens { - match agent::compact_project( - &client, - &root, - &session, - outcome.history, - cancellation, - |event| { - let _ = events.send(event); - }, - ) - .await - { - Ok(kept) => state.replace_history(&root, &session, kept)?, - Err(error) => { - let _ = events.send(agent::AgentEvent::Warning { - text: format!( - "Automatic compaction failed; continuing with the full transcript: {error}" - ), - }); + match persist_agent_outcome(&state, &root, &session, &outcome, |event| { + let _ = events.send(event); + }) { + Ok(true) => { + if prompt_tokens > context_budget_tokens { + match agent::compact_project( + &client, + &root, + &session, + outcome.history, + cancellation, + |event| { + let _ = events.send(event); + }, + ) + .await + { + Ok(kept) => state.replace_history(&root, &session, kept)?, + Err(error) => { + let _ = events.send(agent::AgentEvent::Warning { + text: format!( + "Automatic compaction failed; continuing with the full transcript: {error}" + ), + }); + } + } } + state.clear_active(&root, &session)?; + Ok(()) + } + Ok(false) => { + state.clear_active(&root, &session)?; + Ok(()) + } + Err(error) => { + let _ = state.clear_active(&root, &session); + Err(error) } } - state.clear_active(&root, &session)?; - Ok(()) } Err(BenchError::Cancelled { .. }) => { + let _ = events.send(agent::AgentEvent::TextRollback); state.clear_active(&root, &session)?; Ok(()) } Err(error) => { + let _ = events.send(agent::AgentEvent::TextRollback); state.clear_active(&root, &session)?; Err(error) } @@ -521,7 +561,7 @@ async fn load_render_config( } /// Persist the render panel selection so the agent composes for the same -/// renderer/profile the user renders with. +/// renderer/orchestration the user renders with. #[tauri::command] async fn save_render_config( root: PathBuf, @@ -785,6 +825,104 @@ pub fn run() { mod tests { use super::*; + fn temp_project(name: &str) -> PathBuf { + let root = std::env::temp_dir().join(format!( + "scorebench-lib-{name}-{}-{}", + std::process::id(), + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + std::fs::create_dir_all(&root).unwrap(); + root + } + + #[test] + fn streamed_text_finalizes_only_after_history_persists() { + let root = temp_project("agent-persistence"); + let state = agent::AgentState::default(); + let (root, mut history, _, _) = state.begin(&root, "main", "request".into()).unwrap(); + history.push(llm::types::InputItem::Message { + role: llm::types::InputRole::Assistant, + content: "durable answer".into(), + }); + let outcome = agent::RunOutcome { + history, + prompt_tokens: Some(7), + status: agent::RunStatus::Complete, + }; + let mut events = Vec::new(); + assert!( + persist_agent_outcome(&state, &root, "main", &outcome, |event| events.push(event)) + .unwrap() + ); + assert!(matches!( + events.as_slice(), + [agent::AgentEvent::TextFinalize] + )); + assert!(memory::load_transcript(&root, "main") + .unwrap() + .items + .iter() + .any(|item| matches!( + item, + llm::types::InputItem::Message { + role: llm::types::InputRole::Assistant, + content, + } if content.display_text() == "durable answer" + ))); + + let divergent = agent::RunOutcome { + history: Vec::new(), + prompt_tokens: None, + status: agent::RunStatus::Complete, + }; + let mut failed_events = Vec::new(); + assert!( + persist_agent_outcome(&state, &root, "main", &divergent, |event| { + failed_events.push(event) + }) + .is_err() + ); + assert!(matches!( + failed_events.as_slice(), + [agent::AgentEvent::TextRollback] + )); + + let rolled_back = agent::RunOutcome { + history: vec![llm::types::InputItem::Message { + role: llm::types::InputRole::Assistant, + content: "must not persist".into(), + }], + prompt_tokens: Some(9), + status: agent::RunStatus::MaxTurns, + }; + let mut rollback_events = Vec::new(); + assert!( + !persist_agent_outcome(&state, &root, "main", &rolled_back, |event| { + rollback_events.push(event) + }) + .unwrap() + ); + assert!(matches!( + rollback_events.as_slice(), + [agent::AgentEvent::TextRollback] + )); + assert!(!memory::load_transcript(&root, "main") + .unwrap() + .items + .iter() + .any(|item| matches!( + item, + llm::types::InputItem::Message { + role: llm::types::InputRole::Assistant, + content, + } if content.display_text() == "must not persist" + ))); + std::fs::remove_dir_all(root).unwrap(); + } + #[test] fn recording_filter_follows_suggested_extension() { assert_eq!( diff --git a/src-tauri/src/manifest.rs b/src-tauri/src/manifest.rs index 7864dfb..89bec5e 100644 --- a/src-tauri/src/manifest.rs +++ b/src-tauri/src/manifest.rs @@ -1,16 +1,18 @@ //! `bench.json` project manifest (docs/roadmap.md: plain-file project state) -//! plus the render-profile compatibility check the agent toolchain runs -//! before scorekit's build would fail. +//! plus the orchestration compatibility check the agent toolchain runs +//! before scorekit's sfizz build would fail. //! //! The manifest is written by the GUI render panel and read by the agent //! core, so the agent composes for the same renderer configuration the user //! renders with. Reads are tolerant: a missing or corrupt manifest never //! blocks chat or scene writes. -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet}; +use std::fmt; use std::path::{Path, PathBuf}; -use serde::{Deserialize, Serialize}; +use serde::de::{self, MapAccess, Visitor}; +use serde::{Deserialize, Deserializer, Serialize}; use serde_json::{Map, Value}; use crate::error::BenchError; @@ -25,10 +27,13 @@ pub const MANIFEST_FILE: &str = "bench.json"; pub struct RenderConfig { #[serde(default, skip_serializing_if = "Option::is_none")] pub renderer: Option, - /// Renderer profile path: project-relative when inside the project, - /// absolute otherwise (mirrors the GUI build parameter). + /// Orchestration profile path: project-relative when inside the + /// project, absolute otherwise (mirrors the GUI build parameter). + /// Routes scene track palettes to leaf renderer profiles; only + /// meaningful for `--renderer sfizz` (see `scorekit schema + /// --orchestration`). #[serde(default, skip_serializing_if = "Option::is_none")] - pub profile: Option, + pub orchestration: Option, /// Texture profile path, independent of the synthesizer backend. #[serde(default, skip_serializing_if = "Option::is_none")] pub texture_profile: Option, @@ -99,43 +104,76 @@ pub fn save_style(root: &Path, style: Option) -> Result<(), BenchError save(root, &manifest) } -/// Result of checking one scene against the active renderer profile. +/// One scene track's routing outcome under the active orchestration: which +/// palette it resolved to (declared `palette` or the orchestration's +/// `default_palette`), the leaf renderer profile that palette binds to, and +/// any problem that would make the sfizz build fail for this track. #[derive(Debug, Clone, Serialize)] -pub struct ProfileCompat { - /// Profile path as stored in the manifest. - pub profile: String, +pub struct TrackOrchestrationCompat { + pub track_id: String, + /// Palette this track resolved to (declared, or the orchestration default). + pub palette: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub instrument: Option, + /// Leaf renderer profile path the palette binds to, once resolved. + #[serde(skip_serializing_if = "Option::is_none")] + pub profile: Option, #[serde(skip_serializing_if = "Option::is_none")] pub profile_name: Option, - /// Instrument keys the profile maps (sorted, deduplicated). - pub mapped: Vec, - /// Scene track instruments with no mapping (sorted, deduplicated). - pub unmapped: Vec, - /// Profile file could not be read or parsed. + /// Missing palette, unreadable leaf profile, or unmapped instrument — + /// whichever applies. `None` means this track's routing is sound. + #[serde(skip_serializing_if = "Option::is_none")] + pub error: Option, +} + +/// Result of checking a scene's tracks against the active multi-profile +/// orchestration: per-track palette resolution, leaf profile mapping, and +/// instrument coverage. Replaces the single-profile `ProfileCompat` model +/// now that scenes route tracks through logical palettes. +#[derive(Debug, Clone, Serialize)] +pub struct OrchestrationCompat { + /// Orchestration path as stored in the manifest. + pub orchestration: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub orchestration_name: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub default_palette: Option, + pub tracks: Vec, + /// The orchestration file itself could not be read or parsed. #[serde(skip_serializing_if = "Option::is_none")] pub error: Option, } -impl ProfileCompat { +impl OrchestrationCompat { pub fn is_compatible(&self) -> bool { - self.unmapped.is_empty() && self.error.is_none() + self.error.is_none() && self.tracks.iter().all(|track| track.error.is_none()) } /// One-line report for tool summaries and prompts. pub fn message(&self) -> String { - let name = self.profile_name.as_deref().unwrap_or(&self.profile); + let name = self + .orchestration_name + .as_deref() + .unwrap_or(&self.orchestration); if let Some(error) = &self.error { - return format!("renderer profile `{name}` is unusable: {error}"); + return format!("orchestration `{name}` is unusable: {error}"); } - if self.unmapped.is_empty() { - format!("all track instruments are mapped by renderer profile `{name}`") + let problems: Vec = self + .tracks + .iter() + .filter_map(|track| { + track + .error + .as_ref() + .map(|error| format!("track `{}`: {error}", track.track_id)) + }) + .collect(); + if problems.is_empty() { + format!("all track instruments are mapped by orchestration `{name}`") } else { format!( - "renderer profile `{name}` has no mapping for instrument(s) {}; the sfizz build will fail", - self.unmapped - .iter() - .map(|key| format!("`{key}`")) - .collect::>() - .join(", ") + "orchestration `{name}` has unresolved track(s), the sfizz build will fail: {}", + problems.join("; ") ) } } @@ -153,13 +191,27 @@ pub struct TextureProfileCompat { pub available: Vec, /// Source keys used by the scene but absent from the profile. pub missing: Vec, + /// Structured v0.6 sources declare the scheduling modes they support. + /// Legacy path-only bindings have no declaration and therefore do not + /// participate in this local preflight. + pub mode_mismatches: Vec, #[serde(skip_serializing_if = "Option::is_none")] pub error: Option, } +#[derive(Debug, Clone, Serialize)] +pub struct TextureModeMismatch { + pub source: String, + pub requested: String, + pub available: Vec, +} + impl TextureProfileCompat { pub fn is_compatible(&self) -> bool { - self.profile.is_some() && self.missing.is_empty() && self.error.is_none() + self.profile.is_some() + && self.missing.is_empty() + && self.mode_mismatches.is_empty() + && self.error.is_none() } pub fn message(&self) -> String { @@ -170,45 +222,68 @@ impl TextureProfileCompat { if let Some(error) = &self.error { return format!("texture profile `{name}` is unusable: {error}"); } - if self.missing.is_empty() { - format!("all texture sources are mapped by texture profile `{name}`") - } else { - format!( - "texture profile `{name}` has no mapping for source(s) {}; the scorekit build will fail", + let mut problems = Vec::new(); + if !self.missing.is_empty() { + problems.push(format!( + "texture profile `{name}` has no mapping for source(s) {}", self.missing .iter() .map(|key| format!("`{key}`")) .collect::>() .join(", ") - ) + )); + } + for mismatch in &self.mode_mismatches { + problems.push(format!( + "source `{}` only declares mode(s) {}; requested `{}`", + mismatch.source, + mismatch + .available + .iter() + .map(|mode| format!("`{mode}`")) + .collect::>() + .join(", "), + mismatch.requested + )); + } + if problems.is_empty() { + format!("all texture sources and playback modes are mapped by texture profile `{name}`") + } else { + format!("{}; the scorekit build will fail", problems.join("; ")) } } } -/// The profile only matters for the sfizz backend (SF2 renderers take a -/// soundfont instead), matching scorekit's `--profile` contract. -fn active_profile(render: &RenderConfig) -> Option<&str> { - match (render.renderer.as_deref(), render.profile.as_deref()) { - (Some("sfizz"), Some(profile)) if !profile.trim().is_empty() => Some(profile), +/// Orchestration only matters for the sfizz backend (SF2 renderers take a +/// soundfont instead), matching scorekit's `--orchestration` contract. +fn active_orchestration(render: &RenderConfig) -> Option<&str> { + match (render.renderer.as_deref(), render.orchestration.as_deref()) { + (Some("sfizz"), Some(orchestration)) if !orchestration.trim().is_empty() => { + Some(orchestration) + } _ => None, } } -pub fn resolve_profile_path(root: &Path, profile: &str) -> PathBuf { - let path = Path::new(profile); - if path.is_absolute() { - path.to_path_buf() +/// Resolve a manifest-stored path (orchestration, texture profile, ...) +/// against the project root: project-relative when relative, used as-is +/// when absolute (mirrors the GUI's file picker output). +pub fn resolve_config_path(root: &Path, path: &str) -> PathBuf { + resolve_relative(root, path) +} + +fn resolve_relative(base: &Path, path: &str) -> PathBuf { + let candidate = Path::new(path); + if candidate.is_absolute() { + candidate.to_path_buf() } else { - root.join(path) + base.join(candidate) } } -/// Load the instrument keys a renderer profile maps. This reads the same -/// YAML scorekit loads and compares keys only — no musical semantics. -pub fn profile_instruments( - root: &Path, - profile: &str, -) -> Result<(Option, Vec), String> { +/// Load the instrument keys one leaf renderer profile maps. This reads the +/// same YAML scorekit loads and compares keys only — no musical semantics. +fn load_profile_instruments(path: &Path) -> Result<(Option, Vec), String> { #[derive(Deserialize)] struct ProfileWire { #[serde(default)] @@ -216,66 +291,437 @@ pub fn profile_instruments( #[serde(default)] instruments: BTreeMap, } - let path = resolve_profile_path(root, profile); - let raw = std::fs::read_to_string(&path) + let raw = std::fs::read_to_string(path) .map_err(|error| format!("cannot read `{}`: {error}", path.display()))?; let wire: ProfileWire = serde_yaml::from_str(&raw) .map_err(|error| format!("`{}` is not a renderer profile: {error}", path.display()))?; Ok((wire.name, wire.instruments.into_keys().collect())) } -/// Load the portable source keys a scorekit texture profile exposes. -pub fn texture_profile_sources( - root: &Path, - profile: &str, -) -> Result<(Option, Vec), String> { +/// One palette as declared by an orchestration, with its leaf renderer +/// profile resolved (or the error that made it unusable). +#[derive(Debug, Clone)] +pub struct PaletteInfo { + pub name: String, + /// Leaf profile path exactly as declared in the orchestration. + pub declared_profile: String, + pub profile_name: Option, + pub instruments: Vec, + pub error: Option, +} + +/// An orchestration profile loaded for compatibility checking and prompt +/// injection: its declared name, default palette, and every palette's +/// resolved leaf renderer profile. +#[derive(Debug, Clone)] +pub struct OrchestrationInfo { + pub name: Option, + pub default_palette: Option, + palettes: BTreeMap, +} + +/// Load an orchestration profile and every palette's leaf renderer profile. +/// Leaf `profile` paths are resolved relative to the orchestration file's +/// own directory (scorekit's contract), never the project root. +pub fn load_orchestration(root: &Path, orchestration: &str) -> Result { #[derive(Deserialize)] - struct ProfileWire { + struct PaletteWire { + profile: String, + } + #[derive(Deserialize)] + struct OrchestrationWire { #[serde(default)] name: Option, - sources: BTreeMap, + #[serde(default)] + default_palette: Option, + #[serde(default)] + palettes: BTreeMap, } - let path = resolve_profile_path(root, profile); + let path = resolve_config_path(root, orchestration); let raw = std::fs::read_to_string(&path) .map_err(|error| format!("cannot read `{}`: {error}", path.display()))?; - let wire: ProfileWire = serde_yaml::from_str(&raw) + let wire: OrchestrationWire = serde_yaml::from_str(&raw).map_err(|error| { + format!( + "`{}` is not an orchestration profile: {error}", + path.display() + ) + })?; + let base_dir = path.parent().unwrap_or_else(|| Path::new(".")); + let palettes = wire + .palettes + .into_iter() + .map(|(name, binding)| { + let leaf_path = resolve_relative(base_dir, &binding.profile); + let (profile_name, instruments, error) = match load_profile_instruments(&leaf_path) { + Ok((profile_name, instruments)) => (profile_name, instruments, None), + Err(error) => (None, Vec::new(), Some(error)), + }; + ( + name.clone(), + PaletteInfo { + name, + declared_profile: binding.profile, + profile_name, + instruments, + error, + }, + ) + }) + .collect(); + Ok(OrchestrationInfo { + name: wire.name, + default_palette: wire.default_palette, + palettes, + }) +} + +impl OrchestrationInfo { + /// Palettes in name order, for prompt injection. + pub fn palettes(&self) -> impl Iterator { + self.palettes.values() + } +} + +#[derive(Debug, Clone)] +enum TextureSourceBinding { + LegacyPath(String), + Discoverable(TextureSource), +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(deny_unknown_fields)] +struct TextureSource { + path: String, + description: String, + category: String, + tags: Vec, + playback: TexturePlayback, + use_cases: Vec, + provenance: TextureProvenance, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(deny_unknown_fields)] +struct TexturePlayback { + modes: Vec, + default_mode: String, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(deny_unknown_fields)] +struct TextureProvenance { + library: String, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(deny_unknown_fields)] +struct TextureProfile { + #[serde(default = "default_texture_profile_schema_version")] + schema_version: u16, + name: String, + #[serde(default)] + description: Option, + #[serde(default)] + root: Option, + sources: BTreeMap, +} + +fn default_texture_profile_schema_version() -> u16 { + 1 +} + +impl<'de> Deserialize<'de> for TextureSourceBinding { + fn deserialize(deserializer: D) -> Result + where + D: Deserializer<'de>, + { + struct BindingVisitor; + + impl<'de> Visitor<'de> for BindingVisitor { + type Value = TextureSourceBinding; + + fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + formatter.write_str("an audio path string or a structured texture source") + } + + fn visit_str(self, value: &str) -> Result + where + E: de::Error, + { + Ok(TextureSourceBinding::LegacyPath(value.to_owned())) + } + + fn visit_string(self, value: String) -> Result + where + E: de::Error, + { + Ok(TextureSourceBinding::LegacyPath(value)) + } + + fn visit_map(self, map: M) -> Result + where + M: MapAccess<'de>, + { + TextureSource::deserialize(de::value::MapAccessDeserializer::new(map)) + .map(TextureSourceBinding::Discoverable) + } + } + + deserializer.deserialize_any(BindingVisitor) + } +} + +impl TextureSourceBinding { + fn declared_modes(&self) -> Option<&[String]> { + match self { + Self::LegacyPath(_) => None, + Self::Discoverable(source) => Some(source.playback.modes.as_slice()), + } + } + + fn validate(&self, field: &str) -> Result<(), String> { + match self { + Self::LegacyPath(path) if path.trim().is_empty() => { + Err(format!("{field} audio path must not be empty")) + } + Self::LegacyPath(_) => Ok(()), + Self::Discoverable(source) => source.validate(field), + } + } +} + +impl TextureSource { + fn validate(&self, field: &str) -> Result<(), String> { + if self.path.trim().is_empty() { + return Err(format!("{field}.path must not be empty")); + } + if self.description.trim().is_empty() { + return Err(format!("{field}.description must not be empty")); + } + if !matches!( + self.category.as_str(), + "ambience" + | "foley" + | "impact" + | "transition" + | "tonal" + | "industrial" + | "organic" + | "sound_design" + ) { + return Err(format!( + "{field}.category `{}` is not a ScoreKit category", + self.category + )); + } + validate_texture_tokens(&format!("{field}.tags"), &self.tags)?; + validate_texture_tokens(&format!("{field}.use_cases"), &self.use_cases)?; + if !valid_texture_library_identity(&self.provenance.library) { + return Err(format!( + "{field}.provenance.library `{}` must match @", + self.provenance.library + )); + } + if self.playback.modes.is_empty() { + return Err(format!("{field}.playback.modes must not be empty")); + } + let mut modes = BTreeSet::new(); + for mode in &self.playback.modes { + if !matches!(mode.as_str(), "loop" | "one_shot") { + return Err(format!( + "{field}.playback.modes contains unsupported mode `{mode}`" + )); + } + if !modes.insert(mode) { + return Err(format!( + "{field}.playback.modes contains duplicate mode `{mode}`" + )); + } + } + if !self.playback.modes.contains(&self.playback.default_mode) { + return Err(format!( + "{field}.playback.default_mode `{}` is not listed in modes", + self.playback.default_mode + )); + } + Ok(()) + } +} + +impl TextureProfile { + fn validate(&self) -> Result<(), String> { + if self.schema_version != 1 { + return Err(format!( + "schema_version {} is unsupported; expected 1", + self.schema_version + )); + } + if self.name.trim().is_empty() { + return Err("name must not be empty".into()); + } + if self.sources.is_empty() { + return Err("sources must not be empty".into()); + } + // These fields are optional and intentionally unconstrained by + // ScoreKit v0.6; reading them keeps this mirror explicit. + let _ = (&self.description, &self.root); + for (name, source) in &self.sources { + if !valid_texture_token(name, 64) { + return Err(format!("sources.{name} must match [a-z][a-z0-9_-]{{0,63}}")); + } + source.validate(&format!("sources.{name}"))?; + } + Ok(()) + } +} + +fn valid_texture_token(value: &str, maximum: usize) -> bool { + !value.is_empty() + && value.len() <= maximum + && value.bytes().enumerate().all(|(index, byte)| { + byte.is_ascii_lowercase() + || (index > 0 && (byte.is_ascii_digit() || matches!(byte, b'_' | b'-'))) + }) +} + +fn validate_texture_tokens(field: &str, values: &[String]) -> Result<(), String> { + if values.is_empty() { + return Err(format!("{field} must list at least one entry")); + } + if values.len() > 16 { + return Err(format!("{field} exceeds the maximum of 16 entries")); + } + let mut unique = BTreeSet::new(); + for value in values { + if !valid_texture_token(value, 32) { + return Err(format!( + "{field} entry `{value}` must match [a-z][a-z0-9_-]{{0,31}}" + )); + } + if !unique.insert(value) { + return Err(format!("{field} contains duplicate entry `{value}`")); + } + } + Ok(()) +} + +fn valid_texture_library_identity(identity: &str) -> bool { + let Some((library, version)) = identity.split_once('@') else { + return false; + }; + !library.is_empty() + && !version.is_empty() + && !version.contains('@') + && library.bytes().enumerate().all(|(index, byte)| { + byte.is_ascii_alphanumeric() || (index > 0 && matches!(byte, b'.' | b'_' | b'-' | b'/')) + }) + && version + .bytes() + .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'.' | b'_' | b'+' | b'-')) +} + +fn load_texture_profile(root: &Path, profile: &str) -> Result { + let path = resolve_config_path(root, profile); + let raw = std::fs::read_to_string(&path) + .map_err(|error| format!("cannot read `{}`: {error}", path.display()))?; + let wire: TextureProfile = serde_yaml::from_str(&raw) .map_err(|error| format!("`{}` is not a texture profile: {error}", path.display()))?; - Ok((wire.name, wire.sources.into_keys().collect())) + wire.validate().map_err(|error| { + format!( + "`{}` is not a valid texture profile: {error}", + path.display() + ) + })?; + Ok(wire) } -/// Cross-check a scene's track instruments against the manifest's active -/// renderer profile. `None` when no sfizz profile is configured or the scene -/// YAML does not parse (scorekit validate owns that failure). +/// Load the portable source keys a scorekit texture profile exposes. Both the +/// v0.5 path-only form and v0.6 discoverable object form remain readable. +pub fn texture_profile_sources( + root: &Path, + profile: &str, +) -> Result<(Option, Vec), String> { + let wire = load_texture_profile(root, profile)?; + Ok((Some(wire.name), wire.sources.into_keys().collect())) +} + +/// Cross-check a scene's tracks against the manifest's active orchestration: +/// each track's declared or default palette must exist, that palette's leaf +/// renderer profile must be readable, and the track's instrument must be +/// mapped by it. `None` when no sfizz orchestration is configured or the +/// scene YAML does not parse (scorekit validate owns that failure). pub fn check_scene_profile( root: &Path, scene: &Path, render: &RenderConfig, -) -> Option { - let profile = active_profile(render)?; - let (profile_name, mapped) = match profile_instruments(root, profile) { - Ok(loaded) => loaded, +) -> Option { + let orchestration = active_orchestration(render)?; + let info = match load_orchestration(root, orchestration) { + Ok(info) => info, Err(error) => { - return Some(ProfileCompat { - profile: profile.to_owned(), - profile_name: None, - mapped: Vec::new(), - unmapped: Vec::new(), + return Some(OrchestrationCompat { + orchestration: orchestration.to_owned(), + orchestration_name: None, + default_palette: None, + tracks: Vec::new(), error: Some(error), }); } }; - let instruments = scene_instruments(scene)?; - let mut unmapped: Vec = instruments + let scene_tracks = scene_tracks(scene)?; + let orchestration_label = info.name.as_deref().unwrap_or(orchestration).to_owned(); + let tracks = scene_tracks .into_iter() - .filter(|instrument| !mapped.contains(instrument)) + .map(|track| { + let palette_name = track + .palette + .or_else(|| info.default_palette.clone()) + .unwrap_or_default(); + let Some(palette) = info.palettes.get(&palette_name) else { + return TrackOrchestrationCompat { + track_id: track.id, + palette: palette_name.clone(), + instrument: track.instrument, + profile: None, + profile_name: None, + error: Some(format!( + "orchestration `{orchestration_label}` has no palette `{palette_name}` (defined: {:?})", + info.palettes.keys().collect::>() + )), + }; + }; + let error = if let Some(profile_error) = &palette.error { + Some(format!( + "renderer profile `{}` is unusable: {profile_error}", + palette.declared_profile + )) + } else { + track.instrument.as_ref().and_then(|instrument| { + if palette.instruments.contains(instrument) { + None + } else { + let name = palette.profile_name.as_deref().unwrap_or(&palette.declared_profile); + Some(format!( + "renderer profile `{name}` has no mapping for instrument `{instrument}`" + )) + } + }) + }; + TrackOrchestrationCompat { + track_id: track.id, + palette: palette_name, + instrument: track.instrument, + profile: Some(palette.declared_profile.clone()), + profile_name: palette.profile_name.clone(), + error, + } + }) .collect(); - unmapped.sort_unstable(); - unmapped.dedup(); - Some(ProfileCompat { - profile: profile.to_owned(), - profile_name, - mapped, - unmapped, + Some(OrchestrationCompat { + orchestration: orchestration.to_owned(), + orchestration_name: info.name, + default_palette: info.default_palette, + tracks, error: None, }) } @@ -287,7 +733,7 @@ pub fn check_scene_texture_profile( scene: &Path, render: &RenderConfig, ) -> Option { - let used = scene_texture_sources(scene)?; + let used = scene_texture_uses(scene)?; if used.is_empty() { return None; } @@ -300,11 +746,12 @@ pub fn check_scene_texture_profile( profile: None, profile_name: None, available: Vec::new(), - missing: used, + missing: used.into_iter().map(|texture| texture.source).collect(), + mode_mismatches: Vec::new(), error: None, }); }; - let (profile_name, available) = match texture_profile_sources(root, profile) { + let loaded = match load_texture_profile(root, profile) { Ok(loaded) => loaded, Err(error) => { return Some(TextureProfileCompat { @@ -312,41 +759,84 @@ pub fn check_scene_texture_profile( profile_name: None, available: Vec::new(), missing: Vec::new(), + mode_mismatches: Vec::new(), error: Some(error), }); } }; + let profile_name = loaded.name; + let available = loaded.sources.keys().cloned().collect::>(); let mut missing = used - .into_iter() - .filter(|source| !available.contains(source)) + .iter() + .filter(|texture| !loaded.sources.contains_key(&texture.source)) + .map(|texture| texture.source.clone()) .collect::>(); missing.sort_unstable(); missing.dedup(); + let mode_mismatches = used + .into_iter() + .filter_map(|texture| { + let requested = texture.mode?; + let available = loaded.sources.get(&texture.source)?.declared_modes()?; + (!available.iter().any(|mode| mode == &requested)).then(|| TextureModeMismatch { + source: texture.source, + requested, + available: available.to_vec(), + }) + }) + .collect(); Some(TextureProfileCompat { profile: Some(profile.to_owned()), - profile_name, + profile_name: Some(profile_name), available, missing, + mode_mismatches, error: None, }) } -/// Track instrument names from scene YAML. Tolerant reader: unparseable -/// scenes or tracks without a string instrument yield nothing here because -/// `scorekit validate` is the authority for scene shape errors. -fn scene_instruments(scene: &Path) -> Option> { +/// One scene track's stable-ID identity as read from YAML for compatibility +/// checking. Tolerant reader: tracks without a string `id` are skipped +/// because `scorekit validate` is the authority for scene shape errors. +struct SceneTrack { + id: String, + instrument: Option, + palette: Option, +} + +fn scene_tracks(scene: &Path) -> Option> { let raw = std::fs::read_to_string(scene).ok()?; let value: serde_yaml::Value = serde_yaml::from_str(&raw).ok()?; let tracks = value.get("tracks")?.as_sequence()?; Some( tracks .iter() - .filter_map(|track| track.get("instrument")?.as_str().map(ToOwned::to_owned)) + .filter_map(|track| { + let id = track.get("id")?.as_str()?.to_owned(); + let instrument = track + .get("instrument") + .and_then(|value| value.as_str()) + .map(ToOwned::to_owned); + let palette = track + .get("palette") + .and_then(|value| value.as_str()) + .map(ToOwned::to_owned); + Some(SceneTrack { + id, + instrument, + palette, + }) + }) .collect(), ) } -fn scene_texture_sources(scene: &Path) -> Option> { +struct SceneTextureUse { + source: String, + mode: Option, +} + +fn scene_texture_uses(scene: &Path) -> Option> { let raw = std::fs::read_to_string(scene).ok()?; let value: serde_yaml::Value = serde_yaml::from_str(&raw).ok()?; let Some(textures) = value.get("textures") else { @@ -356,29 +846,44 @@ fn scene_texture_sources(scene: &Path) -> Option> { Some( textures .iter() - .filter_map(|texture| texture.get("source")?.as_str().map(ToOwned::to_owned)) + .filter_map(|texture| { + let source = texture.get("source")?.as_str()?.to_owned(); + let mode = texture + .get("mode") + .and_then(serde_yaml::Value::as_str) + .map(ToOwned::to_owned); + Some(SceneTextureUse { source, mode }) + }) .collect(), ) } #[cfg(test)] mod tests { + use std::sync::atomic::{AtomicU64, Ordering}; + use super::*; + static NEXT_TEMP: AtomicU64 = AtomicU64::new(0); + + /// The wall clock is too coarse to separate two roots created in the same + /// instant, so parallel tests would share a directory and delete each + /// other's files; the counter makes each root unique by construction. fn temp_project() -> PathBuf { let root = std::env::temp_dir().join(format!( - "scorebench-manifest-{}-{}", + "scorebench-manifest-{}-{}-{}", std::process::id(), std::time::SystemTime::now() .duration_since(std::time::UNIX_EPOCH) .unwrap() - .as_nanos() + .as_nanos(), + NEXT_TEMP.fetch_add(1, Ordering::Relaxed) )); std::fs::create_dir_all(&root).unwrap(); root } - fn write_profile(root: &Path) { + fn write_leaf_profile(root: &Path) { std::fs::create_dir_all(root.join("profiles")).unwrap(); std::fs::write( root.join("profiles/open.yaml"), @@ -387,6 +892,15 @@ mod tests { .unwrap(); } + fn write_orchestration(root: &Path) { + write_leaf_profile(root); + std::fs::write( + root.join("hybrid.yaml"), + "schema_version: 1\nname: hybrid-cinematic\ndefault_palette: default\npalettes:\n default: { profile: profiles/open.yaml }\n", + ) + .unwrap(); + } + fn write_texture_profile(root: &Path) { std::fs::create_dir_all(root.join("profiles")).unwrap(); std::fs::write( @@ -399,7 +913,7 @@ mod tests { fn sfizz_render() -> RenderConfig { RenderConfig { renderer: Some("sfizz".into()), - profile: Some("profiles/open.yaml".into()), + orchestration: Some("hybrid.yaml".into()), texture_profile: Some("profiles/forest-textures.yaml".into()), } } @@ -407,7 +921,7 @@ mod tests { fn textured_render() -> RenderConfig { RenderConfig { renderer: Some("fluidsynth".into()), - profile: None, + orchestration: None, texture_profile: Some("profiles/forest-textures.yaml".into()), } } @@ -491,16 +1005,20 @@ mod tests { #[test] fn unmapped_instrument_is_detected() { let root = temp_project(); - write_profile(&root); + write_orchestration(&root); std::fs::write( root.join("scene.yaml"), - "tracks:\n - instrument: piano\n - instrument: choir\n - instrument: choir\n", + "tracks:\n - { id: lead, instrument: piano }\n - { id: pad, instrument: choir }\n", ) .unwrap(); let compat = check_scene_profile(&root, &root.join("scene.yaml"), &sfizz_render()).unwrap(); assert!(!compat.is_compatible()); - assert_eq!(compat.unmapped, vec!["choir"]); - assert_eq!(compat.profile_name.as_deref(), Some("scoredata-open")); + let pad = compat.tracks.iter().find(|t| t.track_id == "pad").unwrap(); + assert!(pad.error.as_deref().unwrap().contains("`choir`")); + assert_eq!(pad.profile_name.as_deref(), Some("scoredata-open")); + let lead = compat.tracks.iter().find(|t| t.track_id == "lead").unwrap(); + assert!(lead.error.is_none()); + assert!(compat.message().contains("`pad`")); assert!(compat.message().contains("`choir`")); std::fs::remove_dir_all(root).unwrap(); } @@ -508,44 +1026,126 @@ mod tests { #[test] fn fully_mapped_scene_is_compatible() { let root = temp_project(); - write_profile(&root); + write_orchestration(&root); std::fs::write( root.join("scene.yaml"), - "tracks:\n - instrument: piano\n - instrument: strings\n", + "tracks:\n - { id: lead, instrument: piano }\n - { id: harmony, instrument: strings }\n", ) .unwrap(); let compat = check_scene_profile(&root, &root.join("scene.yaml"), &sfizz_render()).unwrap(); assert!(compat.is_compatible()); - assert_eq!(compat.mapped, vec!["piano", "strings"]); + assert_eq!(compat.tracks.len(), 2); + assert!(compat + .tracks + .iter() + .all(|t| t.palette == "default" && t.error.is_none())); + std::fs::remove_dir_all(root).unwrap(); + } + + #[test] + fn track_palette_overrides_default_and_missing_palette_is_reported() { + let root = temp_project(); + write_leaf_profile(&root); + std::fs::write( + root.join("hybrid.yaml"), + "schema_version: 1\nname: hybrid-cinematic\ndefault_palette: default\npalettes:\n default: { profile: profiles/open.yaml }\n", + ) + .unwrap(); + std::fs::write( + root.join("scene.yaml"), + "tracks:\n - { id: lead, instrument: piano }\n - { id: solo, instrument: piano, palette: solo }\n", + ) + .unwrap(); + let compat = check_scene_profile(&root, &root.join("scene.yaml"), &sfizz_render()).unwrap(); + assert!(!compat.is_compatible()); + let solo = compat.tracks.iter().find(|t| t.track_id == "solo").unwrap(); + assert_eq!(solo.palette, "solo"); + assert!(solo.error.as_deref().unwrap().contains("no palette `solo`")); + let lead = compat.tracks.iter().find(|t| t.track_id == "lead").unwrap(); + assert_eq!(lead.palette, "default"); + assert!(lead.error.is_none()); + std::fs::remove_dir_all(root).unwrap(); + } + + /// The sound library scorekit documents lives outside the project, with + /// `profiles/orchestrations/.yaml` binding palettes to sibling + /// `../renderers/.yaml` files. Leaf paths must therefore resolve + /// against the orchestration file's own directory — resolving them against + /// the project root would report every palette as unusable. + #[test] + fn leaf_profiles_resolve_against_the_orchestration_directory() { + let root = temp_project(); + let library = root.join("library"); + std::fs::create_dir_all(library.join("profiles/orchestrations")).unwrap(); + std::fs::create_dir_all(library.join("profiles/renderers")).unwrap(); + std::fs::write( + library.join("profiles/renderers/scoredata-chamber.yaml"), + "name: scoredata-chamber\ninstruments:\n cello:\n sustain: cello.sfz\n", + ) + .unwrap(); + std::fs::write( + library.join("profiles/orchestrations/hybrid-cinematic.yaml"), + "schema_version: 1\nname: hybrid-cinematic\ndefault_palette: solo\npalettes:\n solo: { profile: ../renderers/scoredata-chamber.yaml }\n", + ) + .unwrap(); + std::fs::write( + root.join("scene.yaml"), + "tracks:\n - { id: solo_line, instrument: cello }\n", + ) + .unwrap(); + let render = RenderConfig { + renderer: Some("sfizz".into()), + // Absolute, the way the GUI file picker stores an out-of-project library. + orchestration: Some( + library + .join("profiles/orchestrations/hybrid-cinematic.yaml") + .to_string_lossy() + .into_owned(), + ), + texture_profile: None, + }; + let compat = check_scene_profile(&root, &root.join("scene.yaml"), &render).unwrap(); + assert!(compat.is_compatible(), "{}", compat.message()); + let track = &compat.tracks[0]; + assert_eq!(track.palette, "solo"); + assert_eq!(track.profile_name.as_deref(), Some("scoredata-chamber")); std::fs::remove_dir_all(root).unwrap(); } #[test] - fn non_sfizz_or_missing_profile_skips_check() { + fn non_sfizz_or_missing_orchestration_skips_check() { let root = temp_project(); - write_profile(&root); - std::fs::write(root.join("scene.yaml"), "tracks:\n - instrument: choir\n").unwrap(); + write_orchestration(&root); + std::fs::write( + root.join("scene.yaml"), + "tracks:\n - { id: pad, instrument: choir }\n", + ) + .unwrap(); let scene = root.join("scene.yaml"); let fluidsynth = RenderConfig { renderer: Some("fluidsynth".into()), - profile: Some("profiles/open.yaml".into()), + orchestration: Some("hybrid.yaml".into()), texture_profile: None, }; assert!(check_scene_profile(&root, &scene, &fluidsynth).is_none()); - let no_profile = RenderConfig { + let no_orchestration = RenderConfig { renderer: Some("sfizz".into()), - profile: None, + orchestration: None, texture_profile: None, }; - assert!(check_scene_profile(&root, &scene, &no_profile).is_none()); + assert!(check_scene_profile(&root, &scene, &no_orchestration).is_none()); assert!(check_scene_profile(&root, &scene, &RenderConfig::default()).is_none()); std::fs::remove_dir_all(root).unwrap(); } #[test] - fn unreadable_profile_is_reported_not_fatal() { + fn unreadable_orchestration_is_reported_not_fatal() { let root = temp_project(); - std::fs::write(root.join("scene.yaml"), "tracks:\n - instrument: piano\n").unwrap(); + std::fs::write( + root.join("scene.yaml"), + "tracks:\n - { id: lead, instrument: piano }\n", + ) + .unwrap(); let compat = check_scene_profile(&root, &root.join("scene.yaml"), &sfizz_render()).unwrap(); assert!(!compat.is_compatible()); assert!(compat.error.as_deref().unwrap().contains("cannot read")); @@ -553,10 +1153,31 @@ mod tests { std::fs::remove_dir_all(root).unwrap(); } + #[test] + fn unreadable_leaf_profile_is_reported_per_track() { + let root = temp_project(); + std::fs::write( + root.join("hybrid.yaml"), + "schema_version: 1\nname: hybrid-cinematic\ndefault_palette: default\npalettes:\n default: { profile: profiles/missing.yaml }\n", + ) + .unwrap(); + std::fs::write( + root.join("scene.yaml"), + "tracks:\n - { id: lead, instrument: piano }\n", + ) + .unwrap(); + let compat = check_scene_profile(&root, &root.join("scene.yaml"), &sfizz_render()).unwrap(); + assert!(compat.error.is_none(), "orchestration itself is readable"); + assert!(!compat.is_compatible()); + let lead = compat.tracks.iter().find(|t| t.track_id == "lead").unwrap(); + assert!(lead.error.as_deref().unwrap().contains("is unusable")); + std::fs::remove_dir_all(root).unwrap(); + } + #[test] fn unparseable_scene_defers_to_scorekit_validate() { let root = temp_project(); - write_profile(&root); + write_orchestration(&root); std::fs::write(root.join("scene.yaml"), "tracks: [unterminated").unwrap(); assert!(check_scene_profile(&root, &root.join("scene.yaml"), &sfizz_render()).is_none()); std::fs::remove_dir_all(root).unwrap(); @@ -582,6 +1203,77 @@ mod tests { std::fs::remove_dir_all(root).unwrap(); } + #[test] + fn scorekit_06_structured_texture_sources_are_loaded_and_modes_are_enforced() { + let root = temp_project(); + std::fs::create_dir_all(root.join("profiles")).unwrap(); + std::fs::write( + root.join("profiles/forest-textures.yaml"), + "schema_version: 1\nname: forest\nsources:\n birds:\n path: audio/birds.wav\n description: A short flock call\n category: organic\n tags: [wildlife, chirping]\n playback:\n modes: [one_shot]\n default_mode: one_shot\n use_cases: [forest]\n provenance:\n library: field-recordings@1.0.0\n", + ) + .unwrap(); + let (_, sources) = texture_profile_sources(&root, "profiles/forest-textures.yaml").unwrap(); + assert_eq!(sources, vec!["birds"]); + + std::fs::write( + root.join("scene.yaml"), + "textures:\n - { source: birds, mode: loop }\n", + ) + .unwrap(); + let compat = + check_scene_texture_profile(&root, &root.join("scene.yaml"), &textured_render()) + .unwrap(); + assert!(!compat.is_compatible()); + let serialized = serde_json::to_value(&compat).unwrap(); + assert_eq!( + serialized["mode_mismatches"][0]["source"], + serde_json::json!("birds") + ); + assert_eq!( + serialized["mode_mismatches"][0]["requested"], + serde_json::json!("loop") + ); + assert_eq!( + serialized["mode_mismatches"][0]["available"], + serde_json::json!(["one_shot"]) + ); + assert!(compat.message().contains("one_shot")); + std::fs::remove_dir_all(root).unwrap(); + } + + #[test] + fn malformed_scorekit_06_texture_metadata_blocks_compatibility() { + let root = temp_project(); + std::fs::create_dir_all(root.join("profiles")).unwrap(); + std::fs::write( + root.join("profiles/forest-textures.yaml"), + "schema_version: 1\nname: forest\nsources:\n birds:\n path: audio/birds.wav\n description: A short flock call\n category: organic\n tags: [wildlife]\n use_cases: [forest]\n provenance:\n library: field-recordings@1.0.0\n", + ) + .unwrap(); + std::fs::write( + root.join("scene.yaml"), + "textures:\n - { source: birds, mode: loop }\n", + ) + .unwrap(); + + let error = texture_profile_sources(&root, "profiles/forest-textures.yaml").unwrap_err(); + assert!(error.contains("playback"), "{error}"); + let compat = + check_scene_texture_profile(&root, &root.join("scene.yaml"), &textured_render()) + .unwrap(); + assert!(!compat.is_compatible()); + assert!( + compat + .error + .as_deref() + .unwrap_or_default() + .contains("playback"), + "{}", + compat.message() + ); + std::fs::remove_dir_all(root).unwrap(); + } + #[test] fn textured_scene_requires_profile_but_plain_scene_does_not() { let root = temp_project(); diff --git a/src-tauri/src/observation.rs b/src-tauri/src/observation.rs index c736e8e..5bedd16 100644 --- a/src-tauri/src/observation.rs +++ b/src-tauri/src/observation.rs @@ -17,9 +17,9 @@ pub struct SceneInspection { pub scene: Option, pub parse_error: Option, pub validation: ValidationDisplay, - /// Compatibility with the project's persisted render configuration; - /// `None` when no sfizz profile is active. - pub render_profile: Option, + /// Compatibility with the project's persisted orchestration + /// configuration; `None` when no sfizz orchestration is active. + pub orchestration: Option, /// Texture source compatibility; `None` when the scene has no textures. pub texture_profile: Option, pub last_diff: Option, @@ -55,11 +55,16 @@ pub struct SectionDisplay { pub tempo: Option, pub loop_enabled: Option, pub intensity: Option, - pub mute: Vec, + /// Stable track IDs silenced in this section. + pub mute: Vec, } #[derive(Debug, Clone, Default, Serialize, PartialEq)] pub struct TrackDisplay { + /// Stable scene-local track ID (routing, sections, stems). + pub id: Option, + /// Logical orchestration palette; absent uses the orchestration default. + pub palette: Option, pub instrument: Option, pub pattern: Option, pub motif: Option, @@ -104,13 +109,13 @@ pub fn inspect_scene(root: &Path, rel_path: &str) -> Result SceneDisplay { intensity: number(section, "intensity"), mute: sequence(section, "mute") .into_iter() - .filter_map(|value| value.as_u64()) + .filter_map(|value| value.as_str().map(ToOwned::to_owned)) .collect(), }) .collect(), @@ -190,6 +195,8 @@ fn display(mapping: &Mapping) -> SceneDisplay { .into_iter() .filter_map(|value| value.as_mapping()) .map(|track| TrackDisplay { + id: string(track, "id"), + palette: string(track, "palette"), instrument: string(track, "instrument"), pattern: string(track, "pattern"), motif: string(track, "motif"), @@ -326,7 +333,26 @@ mod tests { let scene = display(value.as_mapping().unwrap()); assert_eq!(scene.sections.len(), 4); assert_eq!(scene.sections[2].tempo, Some(132.0)); - assert_eq!(scene.sections[0].mute, vec![3, 4]); + assert_eq!( + scene.sections[0].mute, + vec!["foundation".to_owned(), "pulse".to_owned()] + ); + } + + #[test] + fn parses_track_id_and_palette() { + let value: Value = serde_yaml::from_str(&fixture("forest_suite.yaml")).unwrap(); + let scene = display(value.as_mapping().unwrap()); + assert_eq!(scene.tracks[0].id.as_deref(), Some("lead")); + assert_eq!(scene.tracks[0].palette, None); + + let with_palette: Value = serde_yaml::from_str( + "tracks:\n - { id: solo_violin, palette: solo, instrument: strings, pattern: sustain }\n", + ) + .unwrap(); + let scene = display(with_palette.as_mapping().unwrap()); + assert_eq!(scene.tracks[0].id.as_deref(), Some("solo_violin")); + assert_eq!(scene.tracks[0].palette.as_deref(), Some("solo")); } #[test] diff --git a/src-tauri/src/project.rs b/src-tauri/src/project.rs index cc44cf4..d98c1ea 100644 --- a/src-tauri/src/project.rs +++ b/src-tauri/src/project.rs @@ -200,7 +200,7 @@ fn require_scene_suffix(rel_path: &str) -> Result<(), BenchError> { } /// Starter content for a manually created scene. -pub const SCENE_TEMPLATE: &str = "title: New Scene\ntempo: 100\nkey: C_major\ntime_signature: \"4/4\"\nbars: 8\nloop: true\ntracks:\n - instrument: piano\n pattern: arpeggio\n intensity: 0.5\n"; +pub const SCENE_TEMPLATE: &str = "title: New Scene\ntempo: 100\nkey: C_major\ntime_signature: \"4/4\"\nbars: 8\nloop: true\ntracks:\n - id: lead\n instrument: piano\n pattern: arpeggio\n intensity: 0.5\n"; /// Create a new scene file from the starter template. Refuses to overwrite. pub fn create_scene(root: &Path, rel_path: &str) -> Result { diff --git a/src-tauri/src/scorekit.rs b/src-tauri/src/scorekit.rs index c008777..a25f60d 100644 --- a/src-tauri/src/scorekit.rs +++ b/src-tauri/src/scorekit.rs @@ -1,6 +1,6 @@ //! Subprocess boundary to the `scorekit` CLI. //! -//! Contract (recorded through scorekit 0.4.0, see `tests/fixtures/`): +//! Contract (recorded through scorekit 0.6.0, see `tests/fixtures/`): //! - success: exit 0; `build` writes `.meta.json` as the machine-readable result //! - failure: stderr carries one JSON object `{code, exit_code, field, location, message}` //! - `doctor --json`: stdout JSON report @@ -19,7 +19,11 @@ use crate::error::BenchError; /// Environment variable that pins the scorekit binary explicitly. pub const SCOREKIT_ENV: &str = "SCOREBENCH_SCOREKIT"; -pub const TESTED_SCOREKIT_RANGE: &str = ">=0.3.0, <0.5.0"; +/// scorekit 0.6.0 adds the discoverable texture source protocol and the +/// `texture inspect` / `texture check` machine commands used by the agent. +/// Keeping a single v0.6 floor avoids advertising tools an older CLI cannot +/// execute; the next minor line must be re-recorded before admission. +pub const TESTED_SCOREKIT_RANGE: &str = ">=0.6.0, <0.7.0"; /// Settings-pinned binary path, seeded by the host layer at startup and /// whenever settings are saved. Held here (not re-read from disk) so core @@ -270,6 +274,10 @@ pub fn parse_error_output(stderr: &str, fallback_exit: i32) -> BenchError { field: Option, location: Option, message: String, + /// Present on `code: "resolution"` failures from scorekit ≥0.5: the + /// per-track instrument resolution, including the palette and leaf + /// renderer profile each track routed through. + report: Option>, } for line in stderr.lines().rev() { let line = line.trim(); @@ -283,6 +291,7 @@ pub fn parse_error_output(stderr: &str, fallback_exit: i32) -> BenchError { exit_code: wire.exit_code, field: wire.field, location: wire.location.map(|v| v.to_string()), + report: wire.report, }; } } @@ -296,6 +305,7 @@ pub fn parse_error_output(stderr: &str, fallback_exit: i32) -> BenchError { exit_code: fallback_exit, field: None, location: None, + report: None, } } @@ -369,6 +379,116 @@ pub fn diff(old: &Path, new: &Path) -> Result { }) } +/// Ask scorekit's resolver which exact/fallback instrument bindings a scene +/// would use. This is the deterministic preflight for world instruments and +/// multi-profile palettes; scorebench does not reproduce the resolver. +pub fn inspect_instruments( + scene: &Path, + orchestration: Option<&Path>, + resolver: Option<&Path>, + fallback_mode: Option<&str>, + verbose: bool, +) -> Result { + let mut args = vec![ + "inspect-instruments".into(), + scene.to_string_lossy().into_owned(), + ]; + if let Some(orchestration) = orchestration { + args.extend([ + "--orchestration".into(), + orchestration.to_string_lossy().into_owned(), + ]); + } + if let Some(resolver) = resolver { + args.extend(["--resolver".into(), resolver.to_string_lossy().into_owned()]); + } + if let Some(mode) = fallback_mode { + args.extend(["--fallback-mode".into(), mode.into()]); + } + if verbose { + args.push("--verbose".into()); + } + args.push("--json".into()); + let stdout = run(&args)?; + serde_json::from_str(&stdout).map_err(|err| BenchError::Io { + message: format!("inspect-instruments output was not valid JSON: {err}"), + }) +} + +#[derive(Debug, Clone, Default, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct TextureInspectParams { + pub source: Option, + pub category: Option, + #[serde(default)] + pub tags: Vec, + pub mode: Option, + pub use_case: Option, +} + +impl TextureInspectParams { + fn to_args(&self) -> Vec { + let mut args = Vec::new(); + if let Some(source) = &self.source { + args.extend(["--source".into(), source.clone()]); + } + if let Some(category) = &self.category { + args.extend(["--category".into(), category.clone()]); + } + for tag in &self.tags { + args.extend(["--tag".into(), tag.clone()]); + } + if let Some(mode) = &self.mode { + args.extend(["--mode".into(), mode.clone()]); + } + if let Some(use_case) = &self.use_case { + args.extend(["--use-case".into(), use_case.clone()]); + } + args + } +} + +/// Exact, conjunctive discovery over a v0.6 structured texture profile. +/// `status: "no_match"` is a successful and truthful answer. +pub fn inspect_textures( + profile: &Path, + params: &TextureInspectParams, +) -> Result { + let mut args = vec![ + "texture".into(), + "inspect".into(), + profile.to_string_lossy().into_owned(), + ]; + args.extend(params.to_args()); + args.push("--json".into()); + let stdout = run(&args)?; + serde_json::from_str(&stdout).map_err(|err| BenchError::Io { + message: format!("texture inspect output was not valid JSON: {err}"), + }) +} + +/// Decode and certify every source in a v0.6 structured texture profile. +/// scorekit owns audio inspection and returns a structured `texture_check` +/// failure report when any entry is missing, silent, or undecodable. +pub fn check_texture_profile( + profile: &Path, + sample_rate: Option, +) -> Result { + let mut args = vec![ + "texture".into(), + "check".into(), + profile.to_string_lossy().into_owned(), + ]; + if let Some(sample_rate) = sample_rate { + args.extend(["--sample-rate".into(), sample_rate.to_string()]); + } + args.push("--json".into()); + let stdout = run(&args)?; + serde_json::from_str(&stdout).map_err(|err| BenchError::Io { + message: format!("texture check output was not valid JSON: {err}"), + }) +} + /// Render parameters exposed by the observation panel. Everything optional; /// omitted fields keep scorekit's own defaults. #[derive(Debug, Clone, Default, Deserialize)] @@ -380,7 +500,9 @@ pub struct BuildParams { pub quality: Option, pub stems: Option, pub soundfont: Option, - pub profile: Option, + /// Multi-profile orchestration path (`--renderer sfizz` only): routes + /// scene track palettes to leaf renderer profiles. + pub orchestration: Option, pub texture_profile: Option, } @@ -405,8 +527,8 @@ impl BuildParams { if let Some(soundfont) = &self.soundfont { args.extend(["--soundfont".into(), soundfont.clone()]); } - if let Some(profile) = &self.profile { - args.extend(["--profile".into(), profile.clone()]); + if let Some(orchestration) = &self.orchestration { + args.extend(["--orchestration".into(), orchestration.clone()]); } if let Some(profile) = &self.texture_profile { args.extend(["--texture-profile".into(), profile.clone()]); @@ -483,6 +605,32 @@ mod tests { } } + /// scorekit 0.5 diagnoses an unbuildable scene per track: which palette it + /// routed through, which leaf profile that palette binds to, and why the + /// closest available substitute was rejected. Dropping the report would + /// leave the agent with a bare count of unresolved instruments. + #[test] + fn keeps_resolution_report_from_orchestration_failure() { + let err = parse_error_output(&fixture("error_resolution.json"), 2); + let BenchError::Scorekit { + code, + exit_code, + report, + .. + } = err + else { + panic!("expected Scorekit error"); + }; + assert_eq!(code, "resolution"); + assert_eq!(exit_code, 2); + let report = report.expect("resolution failures carry a report"); + let track = &report["tracks"][0]; + assert_eq!(track["track_id"], "pad"); + assert_eq!(track["palette"], "solo"); + assert_eq!(track["profile"], "scoredata-chamber"); + assert_eq!(track["status"], "missing"); + } + #[test] fn falls_back_on_non_json_stderr() { let err = parse_error_output("segfault or something", 4); @@ -512,9 +660,19 @@ mod tests { value.get("tools").is_some(), "doctor JSON must have `tools`" ); - assert_eq!( - value.get("scorekit_version").and_then(Value::as_str), - Some("0.4.0") + // The fixture is a recording of the contract scorebench is written + // against, so it must come from a release inside the tested range — + // pinning a literal lets the recording rot behind the range. + let recorded = value + .get("scorekit_version") + .and_then(Value::as_str) + .expect("doctor JSON must report scorekit_version"); + let recorded = Version::parse(recorded).expect("recorded version is semantic"); + assert!( + VersionReq::parse(TESTED_SCOREKIT_RANGE) + .unwrap() + .matches(&recorded), + "doctor fixture records scorekit {recorded}, outside {TESTED_SCOREKIT_RANGE}" ); } @@ -524,6 +682,30 @@ mod tests { for key in ["audio", "loop", "sample_rate", "total_samples", "tracks"] { assert!(value.get(key).is_some(), "meta.json must have `{key}`"); } + // scorekit 0.5 reports every track by its stable scene-local ID: the + // meta track list and the instrument resolution both carry it, which is + // what ties a rendered stem back to the scene track that produced it. + let tracks = value["tracks"].as_array().expect("tracks is an array"); + assert!(!tracks.is_empty(), "fixture must record tracks"); + for track in tracks { + for key in ["id", "palette", "instrument", "articulation"] { + assert!( + track.get(key).is_some(), + "meta.json track must have `{key}`: {track}" + ); + } + } + let resolved = value["instrument_resolution"]["tracks"] + .as_array() + .expect("instrument_resolution.tracks is an array"); + assert_eq!(resolved.len(), tracks.len()); + for (resolution, track) in resolved.iter().zip(tracks) { + assert_eq!( + resolution.get("track_id"), + track.get("id"), + "instrument resolution must be keyed by the same track id" + ); + } } #[test] @@ -606,20 +788,20 @@ mod tests { #[test] fn build_params_render_full_arg_set() { let params = BuildParams { - renderer: Some("timidity".into()), + renderer: Some("sfizz".into()), sample_rate: Some(48000), gain: Some(0.7), quality: Some(6), stems: Some(true), soundfont: None, - profile: None, + orchestration: Some("hybrid.yaml".into()), texture_profile: Some("profiles/forest-textures.yaml".into()), }; assert_eq!( params.to_args(), vec![ "--renderer", - "timidity", + "sfizz", "--sample-rate", "48000", "--gain", @@ -627,6 +809,8 @@ mod tests { "--quality", "6", "--stems", + "--orchestration", + "hybrid.yaml", "--texture-profile", "profiles/forest-textures.yaml" ] @@ -637,6 +821,57 @@ mod tests { assert!(BuildParams::default().to_args().is_empty()); } + #[test] + fn texture_inspect_filters_render_as_exact_repeated_flags() { + let params = TextureInspectParams { + source: Some("rain_soft".into()), + category: Some("ambience".into()), + tags: vec!["rain".into(), "soft".into()], + mode: Some("loop".into()), + use_case: Some("night".into()), + }; + assert_eq!( + params.to_args(), + [ + "--source", + "rain_soft", + "--category", + "ambience", + "--tag", + "rain", + "--tag", + "soft", + "--mode", + "loop", + "--use-case", + "night", + ] + .into_iter() + .map(String::from) + .collect::>() + ); + } + + #[test] + fn texture_check_failure_keeps_the_certification_report() { + let error = parse_error_output( + r#"{"code":"texture_check","exit_code":2,"field":null,"location":null,"message":"1 texture source(s) failed certification","report":{"passed":1,"failed":1,"entries":[{"source":"gone","status":"missing"}]}}"#, + 2, + ); + let BenchError::Scorekit { + code, + exit_code, + report, + .. + } = error + else { + panic!("expected structured scorekit error"); + }; + assert_eq!(code, "texture_check"); + assert_eq!(exit_code, 2); + assert_eq!(report.unwrap()["entries"][0]["source"], "gone"); + } + #[test] fn meta_path_follows_scorekit_convention() { assert_eq!( @@ -649,7 +884,7 @@ mod tests { fn handshake_gates_machine_readable_version() { let report = serde_json::json!({ "ready": true, - "scorekit_version": "0.4.0", + "scorekit_version": "0.6.0", "hints": ["install a renderer"] }); let handshake = @@ -657,19 +892,20 @@ mod tests { assert_eq!(handshake.compatible, Some(true)); assert_eq!(handshake.hints, vec!["install a renderer"]); assert_eq!(handshake.source, Some(LocateSource::Path)); - - // 0.3.x stays inside the tested range: both recorded contracts hold. - let floor = handshake_from_report( + // The v0.6 agent contract depends on structured texture discovery and + // certification, so the whole 0.6 line stays inside the tested range. + let patch = handshake_from_report( PathBuf::from("scorekit"), LocateSource::Path, - serde_json::json!({"ready":true,"scorekit_version":"0.3.0","hints":[]}), + serde_json::json!({"ready":true,"scorekit_version":"0.6.2","hints":[]}), ); - assert_eq!(floor.compatible, Some(true)); + assert_eq!(patch.compatible, Some(true)); + // 0.5 has orchestration but not the v0.6 texture inspect/check contract. let outdated = handshake_from_report( PathBuf::from("scorekit"), LocateSource::Settings, - serde_json::json!({"ready":true,"scorekit_version":"0.2.3","hints":[]}), + serde_json::json!({"ready":true,"scorekit_version":"0.5.9","hints":[]}), ); assert_eq!(outdated.compatible, Some(false)); assert!(outdated @@ -677,6 +913,13 @@ mod tests { .unwrap() .contains("outside the tested range")); + let future = handshake_from_report( + PathBuf::from("scorekit"), + LocateSource::Path, + serde_json::json!({"ready":true,"scorekit_version":"0.7.0","hints":[]}), + ); + assert_eq!(future.compatible, Some(false)); + let legacy = handshake_from_report( PathBuf::from("scorekit"), LocateSource::WellKnown, diff --git a/src-tauri/tests/fixtures/capability/after-fail.yaml b/src-tauri/tests/fixtures/capability/after-fail.yaml new file mode 100644 index 0000000..1cddd50 --- /dev/null +++ b/src-tauri/tests/fixtures/capability/after-fail.yaml @@ -0,0 +1,16 @@ +title: After Fail +tempo: 120 +key: C_major +time_signature: "4/4" +bars: 8 +loop: false +motifs: + theme: + - { degree: 5, beats: 1 } + - { degree: 3, beats: 1 } + - { degree: 1, beats: 1 } +tracks: + - { id: lead, instrument: piano, pattern: melody, motif: theme } + - { id: answer, instrument: flute, pattern: melody, motif: theme } + - { id: harmony, instrument: strings, pattern: sustain } + - { id: foundation, instrument: bass, pattern: bass } diff --git a/src-tauri/tests/fixtures/capability/after-pass.yaml b/src-tauri/tests/fixtures/capability/after-pass.yaml new file mode 100644 index 0000000..0700f80 --- /dev/null +++ b/src-tauri/tests/fixtures/capability/after-pass.yaml @@ -0,0 +1,15 @@ +title: After Pass +tempo: 80 +key: D_minor +time_signature: "4/4" +bars: 8 +loop: true +motifs: + theme: + - { degree: 2, beats: 1 } + - { degree: 4, beats: 1 } + - { degree: 6, beats: 1 } + - { degree: 0, beats: 3 } +tracks: + - { id: lead, instrument: piano, pattern: melody, motif: theme } + - { id: harmony, instrument: strings, pattern: sustain } diff --git a/src-tauri/tests/fixtures/capability/before.yaml b/src-tauri/tests/fixtures/capability/before.yaml new file mode 100644 index 0000000..168d3a0 --- /dev/null +++ b/src-tauri/tests/fixtures/capability/before.yaml @@ -0,0 +1,16 @@ +title: Before +tempo: 100 +key: C_major +time_signature: "4/4" +bars: 8 +loop: true +motifs: + theme: + - { degree: 1, beats: 1 } + - { degree: 3, beats: 1 } + - { degree: 5, beats: 1 } + - { degree: 0, beats: 1 } +tracks: + - { id: lead, instrument: piano, pattern: melody, motif: theme } + - { id: harmony, instrument: strings, pattern: sustain } + - { id: foundation, instrument: bass, pattern: bass } diff --git a/src-tauri/tests/fixtures/capability/lonelier.case.yaml b/src-tauri/tests/fixtures/capability/lonelier.case.yaml new file mode 100644 index 0000000..48b85bf --- /dev/null +++ b/src-tauri/tests/fixtures/capability/lonelier.case.yaml @@ -0,0 +1,16 @@ +schema_version: 1 +id: lonelier +intent: Make it lonelier without losing the motif or loop. +before: before.yaml +after_samples: + - id: pass + path: after-pass.yaml + - id: fail + path: after-fail.yaml +assertions: + tempo: decrease + track_count: decrease + melody_rest_ratio: increase + mode: to_minor + loop: preserve + motif_contour: preserve diff --git a/src-tauri/tests/fixtures/doctor.json b/src-tauri/tests/fixtures/doctor.json index b7fcdeb..ff3a067 100644 --- a/src-tauri/tests/fixtures/doctor.json +++ b/src-tauri/tests/fixtures/doctor.json @@ -1 +1 @@ -{"hints":["Platform macos/aarch64: use release asset scorekit-aarch64-apple-darwin.tar.gz.","Install the standard toolchain with `brew install fluid-synth timidity ffmpeg`.","Install optional SFZ rendering with `brew install talkincode/tap/scorekit-sfizz` or build from source with `make sfizz`.","Apple Silicon: scorekit-sfizz builds sfizz_render from source because upstream macOS binaries are x86_64-only.","Run `make install-default-soundfont` when MuseScore_General.sf2 is missing; SFZ rendering still requires a user-supplied library/profile."],"platform":{"arch":"aarch64","os":"macos","release_asset":"scorekit-aarch64-apple-darwin.tar.gz","target":"aarch64-apple-darwin"},"ready":true,"requirements":{"ffmpeg":true,"renderer":true},"scorekit_version":"0.4.0","sound_library":{"default_soundfont":{"path":"/Volumes/WT9/ScoreData/sf2/MuseScore_General.sf2","status":"ok"},"path":"/Volumes/WT9/ScoreData","source":"SCOREKIT_SOUND_LIBRARY_DIR"},"tools":[{"name":"ffmpeg","path":"/opt/homebrew/bin/ffmpeg","role":"audio export","status":"ok","version":"ffmpeg version 8.1.2 Copyright (c) 2000-2026 the FFmpeg developers"},{"name":"fluidsynth","path":"/opt/homebrew/bin/fluidsynth","role":"SF2 renderer","status":"ok","version":"FluidSynth runtime version 2.5.6"},{"name":"timidity","path":"/opt/homebrew/bin/timidity","role":"alternate SF2 renderer","status":"ok","version":"TiMidity++ version 2.15.0"},{"name":"sfizz_render","path":"/Users/wangjuntao/.local/bin/sfizz_render","role":"SFZ renderer","status":"ok","version":"Render a midi file through an SFZ file using the sfizz library."}]} +{"hints":["Platform macos/aarch64: use release asset scorekit-aarch64-apple-darwin.tar.gz.","Install the standard toolchain with `brew install fluid-synth timidity ffmpeg`.","Install optional SFZ rendering with `brew install talkincode/tap/scorekit-sfizz` or build from source with `make sfizz`.","Apple Silicon: scorekit-sfizz builds sfizz_render from source because upstream macOS binaries are x86_64-only.","Run `make install-default-soundfont` when MuseScore_General.sf2 is missing; SFZ rendering still requires a user-supplied library/profile."],"platform":{"arch":"aarch64","os":"macos","release_asset":"scorekit-aarch64-apple-darwin.tar.gz","target":"aarch64-apple-darwin"},"ready":true,"requirements":{"ffmpeg":true,"renderer":true},"scorekit_version":"0.6.0","sound_library":{"default_soundfont":{"path":"/Volumes/WT9/ScoreData/sf2/MuseScore_General.sf2","status":"ok"},"path":"/Volumes/WT9/ScoreData","source":"SCOREKIT_SOUND_LIBRARY_DIR"},"tools":[{"name":"ffmpeg","path":"/opt/homebrew/bin/ffmpeg","role":"audio export","status":"ok","version":"ffmpeg version 8.1.2 Copyright (c) 2000-2026 the FFmpeg developers"},{"name":"fluidsynth","path":"/opt/homebrew/bin/fluidsynth","role":"SF2 renderer","status":"ok","version":"FluidSynth runtime version 2.5.6"},{"name":"timidity","path":"/opt/homebrew/bin/timidity","role":"alternate SF2 renderer","status":"ok","version":"TiMidity++ version 2.15.0"},{"name":"sfizz_render","path":"/Users/wangjuntao/.local/bin/sfizz_render","role":"SFZ renderer","status":"ok","version":"Render a midi file through an SFZ file using the sfizz library."}]} diff --git a/src-tauri/tests/fixtures/error_resolution.json b/src-tauri/tests/fixtures/error_resolution.json new file mode 100644 index 0000000..0888482 --- /dev/null +++ b/src-tauri/tests/fixtures/error_resolution.json @@ -0,0 +1 @@ +{"code":"resolution","exit_code":2,"message":"1 unresolved instrument(s) in `unmapped.yaml`","report":{"fallbacks":[],"missing_instruments":["choir"],"mode":"conservative","summary":{"alias":0,"exact":0,"fallback":0,"missing":1,"rejected":0},"tracks":[{"best_candidate":{"instrument":"cello","reasons":["compatible_range","articulation_supported","envelope_similar","role_compatible"],"rejected":"strings_fallback_requires_explicit_allowance","score":0.8029999732971191},"canonical":"choir","palette":"solo","profile":"scoredata-chamber","profile_path":"../renderers/scoredata-chamber.yaml","reasons":[],"requested":"choir","requested_articulation":"sustain","resolved":null,"score":0.0,"status":"missing","track":0,"track_id":"pad","warnings":["requested_instrument_not_available","no_candidate_satisfies_fallback_policy"]}]}} diff --git a/src-tauri/tests/fixtures/forest.meta.json b/src-tauri/tests/fixtures/forest.meta.json index aea577b..9e5b743 100644 --- a/src-tauri/tests/fixtures/forest.meta.json +++ b/src-tauri/tests/fixtures/forest.meta.json @@ -17,49 +17,53 @@ { "canonical": "strings", "reasons": [ - "available" + "general_midi_exact" ], "requested": "strings", "resolved": "strings", "score": 1.0, "status": "exact", "track": 0, + "track_id": "harmony", "warnings": [] }, { "canonical": "piano", "reasons": [ - "available" + "general_midi_exact" ], "requested": "piano", "resolved": "piano", "score": 1.0, "status": "exact", "track": 1, + "track_id": "motion", "warnings": [] }, { "canonical": "bass", "reasons": [ - "available" + "general_midi_exact" ], "requested": "bass", "resolved": "bass", "score": 1.0, "status": "exact", "track": 2, + "track_id": "foundation", "warnings": [] }, { "canonical": "drums", "reasons": [ - "available" + "general_midi_exact" ], "requested": "drums", "resolved": "drums", "score": 1.0, "status": "exact", "track": 3, + "track_id": "pulse", "warnings": [] } ] @@ -77,24 +81,36 @@ "total_samples": 920347, "tracks": [ { + "articulation": "sustain", + "id": "harmony", "instrument": "strings", "intensity": 0.4000000059604645, + "palette": null, "pattern": "sustain" }, { + "articulation": "sustain", + "id": "motion", "instrument": "piano", "intensity": 0.550000011920929, + "palette": null, "pattern": "arpeggio" }, { + "articulation": "sustain", + "id": "foundation", "instrument": "bass", "intensity": 0.44999998807907104, + "palette": null, "pattern": "bass" }, { + "articulation": "sustain", + "id": "pulse", "instrument": "drums", "intensity": 0.30000001192092896, + "palette": null, "pattern": "drums" } ] -} \ No newline at end of file +} diff --git a/src-tauri/tests/fixtures/scenes/forest.yaml b/src-tauri/tests/fixtures/scenes/forest.yaml index dd0fdb8..ca16f39 100644 --- a/src-tauri/tests/fixtures/scenes/forest.yaml +++ b/src-tauri/tests/fixtures/scenes/forest.yaml @@ -5,15 +5,19 @@ time_signature: "4/4" bars: 8 loop: true tracks: - - instrument: strings + - id: harmony + instrument: strings pattern: sustain intensity: 0.4 - - instrument: piano + - id: motion + instrument: piano pattern: arpeggio intensity: 0.55 - - instrument: bass + - id: foundation + instrument: bass pattern: bass intensity: 0.45 - - instrument: drums + - id: pulse + instrument: drums pattern: drums intensity: 0.3 diff --git a/src-tauri/tests/fixtures/scenes/forest_suite.yaml b/src-tauri/tests/fixtures/scenes/forest_suite.yaml index 0f17d68..1394ef1 100644 --- a/src-tauri/tests/fixtures/scenes/forest_suite.yaml +++ b/src-tauri/tests/fixtures/scenes/forest_suite.yaml @@ -4,13 +4,25 @@ key: D_minor time_signature: 4/4 bars: 8 unknown_future_field: tolerated +motifs: + theme: + - { degree: 1, beats: 1.5 } + - { degree: 3, beats: 0.5 } + - { degree: 5, beats: 1 } + - { degree: 4, beats: 1 } + - { degree: 3, beats: 1.5 } + - { degree: 2, beats: 0.5 } + - { degree: 1, beats: 2 } tracks: - - { instrument: flute, pattern: melody, motif: theme, intensity: 0.7 } - - { instrument: strings, pattern: sustain, intensity: 0.5 } + - { id: lead, instrument: flute, pattern: melody, motif: theme, intensity: 0.7 } + - { id: harmony, instrument: strings, pattern: sustain, intensity: 0.5 } + - { id: motion, instrument: harp, pattern: arpeggio, intensity: 0.5 } + - { id: foundation, instrument: bass, pattern: bass, intensity: 0.5 } + - { id: pulse, instrument: drums, pattern: drums, intensity: 0.4 } sections: - name: intro bars: 4 - mute: [3, 4] + mute: [foundation, pulse] intensity: 0.7 - name: explore bars: 8 @@ -22,4 +34,4 @@ sections: intensity: 1.4 - name: victory bars: 2 - mute: [4] + mute: [pulse] diff --git a/src/lib/api.test.ts b/src/lib/api.test.ts index cb7fe2b..4157aee 100644 --- a/src/lib/api.test.ts +++ b/src/lib/api.test.ts @@ -1,7 +1,14 @@ import { invoke } from "@tauri-apps/api/core"; import { beforeEach, describe, expect, it, vi } from "vitest"; -import { api, errorText, type SceneInspection } from "./api"; +import { + advanceAgentDraft, + api, + errorText, + isCurrentAgentRun, + resolvedPaletteForTrack, + type SceneInspection, +} from "./api"; vi.mock("@tauri-apps/api/core", () => ({ invoke: vi.fn(), @@ -41,6 +48,108 @@ describe("errorText", () => { 'LLM endpoint returned HTTP 400\n{"error":{"message":"tools[0].strict is unsupported"}}', ); }); + + it("names the track, palette and leaf profile behind an unresolved orchestration", () => { + expect( + errorText({ + kind: "scorekit", + code: "resolution", + exit_code: 2, + message: "1 unresolved instrument(s) in `forest.yaml`", + report: { + tracks: [ + { + track_id: "harmony", + palette: "default", + profile: "scoredata-open", + requested: "strings", + resolved: "strings", + status: "exact", + }, + { + track_id: "pad", + palette: "solo", + profile: "scoredata-chamber", + requested: "choir", + resolved: null, + status: "missing", + best_candidate: { + instrument: "cello", + rejected: "strings_fallback_requires_explicit_allowance", + }, + }, + ], + }, + }), + ).toBe( + "resolution: 1 unresolved instrument(s) in `forest.yaml`\n" + + "track `pad` (palette `solo`, profile `scoredata-chamber`): `choir` unavailable" + + " — closest `cello` rejected (strings_fallback_requires_explicit_allowance)", + ); + }); + + it("leaves errors without a report untouched", () => { + expect( + errorText({ kind: "scorekit", code: "validation", message: "bad scene" }), + ).toBe("validation: bad scene"); + }); +}); + +describe("resolvedPaletteForTrack", () => { + const orchestration = { + orchestration: "hybrid.yaml", + default_palette: "solo", + tracks: [ + { + track_id: "lead", + palette: "solo", + instrument: "cello", + profile: "chamber.yaml", + }, + ], + }; + + it("uses scorekit's resolved palette for a palette-less track", () => { + expect(resolvedPaletteForTrack({ id: "lead", palette: null }, orchestration)).toBe("solo"); + }); + + it("falls back to the authored palette when compatibility data is unavailable", () => { + expect(resolvedPaletteForTrack({ id: "lead", palette: "ensemble" }, null)).toBe("ensemble"); + }); +}); + +describe("advanceAgentDraft", () => { + it("streams deltas provisionally and commits only on an explicit boundary", () => { + let state = advanceAgentDraft("", { type: "text", text: "First" }); + state = advanceAgentDraft(state.pending, { type: "text", text: " second" }); + expect(state).toEqual({ pending: "First second" }); + + state = advanceAgentDraft(state.pending, { type: "text_commit" }); + expect(state).toEqual({ pending: "", committed: "First second" }); + }); + + it("drops an unverified success claim when the backend rejects the turn", () => { + const streamed = advanceAgentDraft("", { type: "text", text: "Done." }); + expect(advanceAgentDraft(streamed.pending, { type: "text_discard" })).toEqual({ + pending: "", + }); + }); +}); + +describe("isCurrentAgentRun", () => { + const run = { id: 7, root: "/projects/a", session: "main" }; + + it("accepts events only for the exact active run, project and session", () => { + expect(isCurrentAgentRun(run, run, "/projects/a", "main")).toBe(true); + }); + + it("rejects late events after a project, session or run switch", () => { + expect(isCurrentAgentRun(run, run, "/projects/b", "main")).toBe(false); + expect(isCurrentAgentRun(run, run, "/projects/a", "other")).toBe(false); + expect( + isCurrentAgentRun(run, { id: 8, root: "/projects/a", session: "main" }, "/projects/a", "main"), + ).toBe(false); + }); }); describe("api.inspectScene", () => { diff --git a/src/lib/api.ts b/src/lib/api.ts index 9784856..02cbfa8 100644 --- a/src/lib/api.ts +++ b/src/lib/api.ts @@ -21,6 +21,26 @@ export interface ProjectInfo { assets: AssetEntry[]; } +/** + * One track's entry in scorekit's instrument-resolution report (≥0.5). It names + * the stable track id, the logical palette the track routed through, and the + * leaf renderer profile that palette binds to. + */ +export interface ResolutionTrack { + track_id?: string | null; + palette?: string | null; + profile?: string | null; + requested?: string | null; + resolved?: string | null; + status?: string | null; + best_candidate?: { instrument?: string | null; rejected?: string | null } | null; +} + +/** `report` payload attached to scorekit `resolution` failures. */ +export interface ResolutionReport { + tracks?: ResolutionTrack[]; +} + export interface BenchError { kind: | "scorekit_missing" @@ -29,7 +49,8 @@ export interface BenchError { | "invalid_project" | "llm" | "cancelled" - | "settings"; + | "settings" + | "agent"; message: string; code?: string; exit_code?: number; @@ -38,6 +59,8 @@ export interface BenchError { status?: number | null; retry_after?: string | null; body_excerpt?: string | null; + /** Structured diagnosis; resolution tracks are rendered specially when present. */ + report?: ResolutionReport | null; } export interface Settings { @@ -105,10 +128,15 @@ export interface SectionDisplay { tempo?: number | null; loop_enabled?: boolean | null; intensity?: number | null; - mute: number[]; + /** Stable track IDs silenced in this section. */ + mute: string[]; } export interface TrackDisplay { + /** Stable scene-local track ID (routing, sections, stems). */ + id?: string | null; + /** Logical orchestration palette; absent uses the orchestration default. */ + palette?: string | null; instrument?: string | null; pattern?: string | null; motif?: string | null; @@ -143,7 +171,7 @@ export interface SceneInspection { scene?: SceneDisplay | null; parse_error?: string | null; validation: ValidationDisplay; - render_profile?: RenderProfileCompat | null; + orchestration?: OrchestrationCompat | null; texture_profile?: TextureProfileCompat | null; last_diff?: unknown; } @@ -156,25 +184,51 @@ export interface ValidationDisplay { /** Matches the Rust `manifest::RenderConfig` serde shape. */ export interface RenderConfig { renderer?: string | null; - profile?: string | null; + orchestration?: string | null; texture_profile?: string | null; } -/** Matches the Rust `manifest::ProfileCompat` serde shape. */ -export interface RenderProfileCompat { - profile: string; +/** Matches the Rust `manifest::TrackOrchestrationCompat` serde shape. */ +export interface TrackOrchestrationCompat { + track_id: string; + palette: string; + instrument?: string | null; + profile?: string | null; profile_name?: string | null; - mapped: string[]; - unmapped: string[]; error?: string | null; } +/** Matches the Rust `manifest::OrchestrationCompat` serde shape. */ +export interface OrchestrationCompat { + orchestration: string; + orchestration_name?: string | null; + default_palette?: string | null; + tracks: TrackOrchestrationCompat[]; + error?: string | null; +} + +/** Resolve the palette scorekit actually routes a scene track through. */ +export function resolvedPaletteForTrack( + track: Pick, + orchestration: OrchestrationCompat | null | undefined, +): string | null { + const routed = track.id + ? orchestration?.tracks.find((candidate) => candidate.track_id === track.id) + : null; + return routed?.palette || track.palette || orchestration?.default_palette || null; +} + /** Matches the Rust `manifest::TextureProfileCompat` serde shape. */ export interface TextureProfileCompat { profile?: string | null; profile_name?: string | null; available: string[]; missing: string[]; + mode_mismatches?: { + source: string; + requested: string; + available: string[]; + }[]; error?: string | null; } @@ -199,6 +253,10 @@ export interface VersionInfo { export type AgentEvent = | { type: "text"; text: string } + | { type: "text_commit" } + | { type: "text_discard" } + | { type: "text_finalize" } + | { type: "text_rollback" } | { type: "warning"; text: string } | { type: "compacted"; turns: number } | { type: "tool_start"; name: string; detail: string } @@ -206,6 +264,45 @@ export type AgentEvent = | { type: "tool_err"; name: string; error: BenchError } | { type: "done" }; +export interface AgentRunIdentity { + id: number; + root: string; + session: string; +} + +/** Reject late Channel events after a project/session/run identity changes. */ +export function isCurrentAgentRun( + expected: AgentRunIdentity, + active: AgentRunIdentity | null, + projectRoot: string | null | undefined, + session: string | null | undefined, +): boolean { + return ( + active?.id === expected.id && + active.root === expected.root && + active.session === expected.session && + projectRoot === expected.root && + session === expected.session + ); +} + +export interface AgentDraftTransition { + pending: string; + committed?: string; +} + +/** Keep streamed model text provisional until the backend accepts the turn. */ +export function advanceAgentDraft( + pending: string, + event: Extract, +): AgentDraftTransition { + if (event.type === "text") return { pending: pending + event.text }; + if (event.type === "text_commit") { + return pending ? { pending: "", committed: pending } : { pending: "" }; + } + return { pending: "" }; +} + export interface BuildResult { output: string; meta_path: string; @@ -225,7 +322,7 @@ export interface BuildParams { quality?: number; stems?: boolean; soundfont?: string; - profile?: string; + orchestration?: string; texture_profile?: string; } @@ -279,11 +376,38 @@ function inspectScene(root: string, relPath: string, revision: number): Promise< return request; } +/** + * scorekit reports an unbuildable orchestration as a count ("2 unresolved + * instrument(s)"); the actionable part — which track, which palette, which leaf + * profile — lives in the attached report, so unresolved tracks are folded into + * the displayed text instead of being dropped. + */ +function unresolvedTrackLines(report: ResolutionReport | null | undefined): string[] { + return (report?.tracks ?? []) + .filter((track) => track.status !== "exact" && track.status !== "alias" && track.resolved == null) + .map((track) => { + const where = [ + track.palette ? `palette \`${track.palette}\`` : null, + track.profile ? `profile \`${track.profile}\`` : null, + ].filter(Boolean); + const candidate = track.best_candidate?.instrument + ? ` — closest \`${track.best_candidate.instrument}\`${ + track.best_candidate.rejected ? ` rejected (${track.best_candidate.rejected})` : "" + }` + : ""; + return `track \`${track.track_id ?? "?"}\`${where.length ? ` (${where.join(", ")})` : ""}: \`${ + track.requested ?? "?" + }\` unavailable${candidate}`; + }); +} + export function errorText(err: unknown): string { const e = err as BenchError; if (e && typeof e === "object" && "message" in e) { const message = e.code ? `${e.code}: ${e.message}` : e.message; - return e.body_excerpt ? `${message}\n${e.body_excerpt}` : message; + const lines = [message, ...unresolvedTrackLines(e.report)]; + if (e.body_excerpt) lines.push(e.body_excerpt); + return lines.join("\n"); } return String(err); } diff --git a/src/lib/components/Chat.svelte b/src/lib/components/Chat.svelte index f4d7eef..6f28677 100644 --- a/src/lib/components/Chat.svelte +++ b/src/lib/components/Chat.svelte @@ -1,6 +1,12 @@ @@ -288,13 +357,14 @@ {:else}
{#each bench.messages as message} -
+
{#if message.role === "tool"} {message.text} {#if message.detail}
{t("chat.output")}
{message.detail}
{/if} {:else} {message.text} + {#if message.provisional}{t("chat.pendingVerification")}{/if} {#if message.attachments?.length} {#each message.attachments as name}⎘ {name}{/each} @@ -303,7 +373,9 @@ {/if}
{/each} - {#if bench.agentBusy}
{/if} + {#if bench.agentBusy && !bench.messages.some((message) => message.provisional)} +
+ {/if}
{/if}
@@ -406,6 +478,8 @@ .msg { max-width: 82%; padding: 9px 12px; border-radius: 9px; font-size: 12px; line-height: 1.55; white-space: pre-wrap; word-break: break-word; } .msg.user { align-self: flex-end; background: var(--accent-soft); border: 1px solid var(--accent-line); } .msg.agent { align-self: flex-start; background: var(--panel-raised); border: 1px solid var(--line); } + .msg.agent.provisional { display: flex; flex-direction: column; gap: 6px; border-style: dashed; } + .msg.agent.provisional small { color: var(--fg-dim); font: 9px var(--mono); letter-spacing: .04em; text-transform: uppercase; } .msg.tool { align-self: stretch; max-width: 100%; color: var(--fg-dim); background: var(--control-bg); border: 1px solid var(--line); border-radius: 5px; font: 11px var(--mono); } .msg.tool .dot { display: inline-block; width: 5px; height: 5px; margin-right: 7px; border-radius: 50%; background: var(--fg-muted); vertical-align: 1px; } .msg.tool.run .dot { background: var(--warning); box-shadow: 0 0 7px var(--warning); } diff --git a/src/lib/components/ScenePanel.svelte b/src/lib/components/ScenePanel.svelte index 8941fc1..cff479f 100644 --- a/src/lib/components/ScenePanel.svelte +++ b/src/lib/components/ScenePanel.svelte @@ -19,7 +19,7 @@ let quality = $state(5); let stems = $state(false); let format = $state<"ogg" | "wav">("ogg"); - let profilePath = $state(null); + let orchestrationPath = $state(null); let textureProfilePath = $state(null); let renderConfigRoot: string | null = null; let inspection = $state(null); @@ -79,7 +79,7 @@ ([config]) => { if (renderConfigRoot !== root) return; renderer = config?.renderer ?? "fluidsynth"; - profilePath = absoluteConfigPath(root, config?.profile); + orchestrationPath = absoluteConfigPath(root, config?.orchestration); textureProfilePath = absoluteConfigPath(root, config?.texture_profile); }, (error) => { @@ -93,9 +93,9 @@ function persistRenderConfig() { const root = bench.project?.root; if (!root) return; - const profile = manifestConfigPath(root, profilePath); + const orchestration = manifestConfigPath(root, orchestrationPath); const texture_profile = manifestConfigPath(root, textureProfilePath); - void api.saveRenderConfig(root, { renderer, profile, texture_profile }).then( + void api.saveRenderConfig(root, { renderer, orchestration, texture_profile }).then( () => bench.projectRevision++, (error) => { bench.buildFailed = true; @@ -172,20 +172,20 @@ } } - const needsProfile = $derived(renderer === "sfizz" && !profilePath); + const needsOrchestration = $derived(renderer === "sfizz" && !orchestrationPath); const needsTextureProfile = $derived( (inspection?.scene?.textures.length ?? 0) > 0 && !textureProfilePath, ); - async function pickProfile() { + async function pickOrchestration() { const picked = await open({ multiple: false, directory: false, defaultPath: bench.project?.root, - filters: [{ name: "Render profile", extensions: ["yaml", "yml"] }], + filters: [{ name: "Orchestration profile", extensions: ["yaml", "yml"] }], }); if (typeof picked === "string") { - profilePath = picked; + orchestrationPath = picked; persistRenderConfig(); } } @@ -208,7 +208,7 @@ !bench.project || !bench.selectedScene || bench.building || - needsProfile || + needsOrchestration || needsTextureProfile ) return; @@ -216,9 +216,9 @@ bench.buildFailed = false; bench.buildStatus = "starting…"; const params: BuildParams = { renderer, sample_rate: sampleRate, gain, quality, stems }; - if (renderer === "sfizz" && profilePath) { + if (renderer === "sfizz" && orchestrationPath) { // Absolute path: the scorekit subprocess does not run from the project root. - params.profile = profilePath; + params.orchestration = orchestrationPath; } if (textureProfilePath) params.texture_profile = textureProfilePath; try { @@ -335,14 +335,13 @@ 
{JSON.stringify(inspection.last_diff, null, 2)}
{/if} - {#if inspection.render_profile} - {#if inspection.render_profile.error} -

{t("panel.profileUnusable", { error: inspection.render_profile.error })}

- {:else if inspection.render_profile.unmapped.length} -

{t("panel.profileUnmapped", { - profile: inspection.render_profile.profile_name ?? inspection.render_profile.profile, - instruments: inspection.render_profile.unmapped.join(", "), - })}

+ {#if inspection.orchestration} + {#if inspection.orchestration.error} +

{t("panel.orchestrationUnusable", { error: inspection.orchestration.error })}

+ {:else} + {#each inspection.orchestration.tracks.filter((track) => track.error) as track} +

{t("panel.orchestrationTrackProblem", { track: track.track_id, error: track.error ?? "" })}

+ {/each} {/if} {/if} {#if inspection.texture_profile} @@ -355,6 +354,14 @@ profile: inspection.texture_profile.profile_name ?? inspection.texture_profile.profile, sources: inspection.texture_profile.missing.join(", "), })}

+ {:else} + {#each inspection.texture_profile.mode_mismatches ?? [] as mismatch} +

{t("panel.textureModeMismatch", { + source: mismatch.source, + requested: mismatch.requested, + available: mismatch.available.join(", "), + })}

+ {/each} {/if} {/if} {#if inspection.validation.error}

{errorText(inspection.validation.error)}

{/if} @@ -378,22 +385,22 @@ {#if renderer === "sfizz"}
- {t("panel.sfzProfile")} + {t("panel.orchestrationProfile")}
- - {#if profilePath} - + {#if orchestrationPath} + {/if}
- {#if needsProfile} -

{t("panel.profileHint")}

+ {#if needsOrchestration} +

{t("panel.orchestrationHint")}

{/if} {/if}
@@ -414,14 +421,13 @@ {#if needsTextureProfile}

{t("panel.textureProfileHint")}

{/if} - {#if renderer === "sfizz" && inspection?.render_profile} - {#if inspection.render_profile.error} -

{t("panel.profileUnusable", { error: inspection.render_profile.error })}

- {:else if inspection.render_profile.unmapped.length} -

{t("panel.profileUnmapped", { - profile: inspection.render_profile.profile_name ?? inspection.render_profile.profile, - instruments: inspection.render_profile.unmapped.join(", "), - })}

+ {#if renderer === "sfizz" && inspection?.orchestration} + {#if inspection.orchestration.error} +

{t("panel.orchestrationUnusable", { error: inspection.orchestration.error })}

+ {:else} + {#each inspection.orchestration.tracks.filter((track) => track.error) as track} +

{t("panel.orchestrationTrackProblem", { track: track.track_id, error: track.error ?? "" })}

+ {/each} {/if} {/if} {#if inspection?.texture_profile} @@ -434,9 +440,17 @@ profile: inspection.texture_profile.profile_name ?? inspection.texture_profile.profile ?? "—", sources: inspection.texture_profile.missing.join(", "), })}

+ {:else} + {#each inspection.texture_profile.mode_mismatches ?? [] as mismatch} +

{t("panel.textureModeMismatch", { + source: mismatch.source, + requested: mismatch.requested, + available: mismatch.available.join(", "), + })}

+ {/each} {/if} {/if} - {#if bench.building}
{/if} diff --git a/src/lib/components/ScenePreview.svelte b/src/lib/components/ScenePreview.svelte index 21fd420..1510815 100644 --- a/src/lib/components/ScenePreview.svelte +++ b/src/lib/components/ScenePreview.svelte @@ -1,5 +1,5 @@