diff --git a/CHANGELOG.md b/CHANGELOG.md index c4efa8737..5bb6498e4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ * Added NSF player and export. * Added stems conversion: to mix several recordings into one reconstruction. +* Changed drive to reach for louder instructions while a recording converts; reconvert anything converted at a drive other than `1.00`. * Improved clarity of reconstructions. * Optimized the size of reconstructions. * Bumped the reconstruction data-version to `2.2` with backward compatibility for `2.1`. diff --git a/README.md b/README.md index defacb2cb..9f8b23777 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@
SampleToNES -

SampleToNES v0.3.2

+

SampleToNES

## Overview @@ -21,26 +21,31 @@ The core idea is to approximate an audio sample using only the chip's basic osci A built-in sequencer lets you arrange the reconstructed samples into patterns and play them back inside the application, so you can experiment with the results before exporting the instruments into FamiTracker. -It supports: +With it you can: -* loading common audio formats: WAV, MP3, FLAC, OGG, AIFF, and AU -* a wide range of NES frequencies, from 15 Hz to 300 Hz, including the two most common standards: - * NTSC (60 Hz) - * PAL (50 Hz) -* various sample rates, from 8000 Hz to 192,000 Hz -* restricting the reconstruction to a chosen subset of oscillators: - * `pulse1` - * `pulse2` - * `triangle` - * `noise` -* exporting reconstructed audio as FamiTracker `.fti` instruments, Bitphase `.json` instrument presets, `.nsf` programs the NES itself plays, or `.wav` +* convert your own recordings — WAV, MP3, FLAC, OGG, AIFF or AU — into NES instruments +* choose which of the four channels each recording may use, and how loud it plays on them +* arrange the results into a song and play it back in the app +* export instruments and songs for: + * [_FamiTracker_](http://famitracker.com/) + * [_Bitphase_](https://bitphase.app/) + * `.nsf` program + * audio `.wav` file ## Installation You can install _SampleToNES_ in three ways: - **Download a release** for Windows or Linux from the [releases page](https://github.com/JakimPL/SampleToNES/releases), extract it, and start `sampletones`. -- **Install from PyPI** on Windows, macOS or Linux. You need Python 3.12 or newer: +- **Install from PyPI** on Windows, macOS or Linux. You need Python 3.12 or newer. On Linux and + macOS, install the audio and file dialog libraries first: + + ```sh + sudo apt-get install libportaudio2 libasound2 python3-tk # Debian and Ubuntu + brew install portaudio # macOS + ``` + + Then install the app and start it: ```sh uv tool install sampletones # or: pipx install sampletones @@ -53,26 +58,12 @@ An NVIDIA graphics card can speed up conversion. The [installation guide](https: ## Usage -### Where your files are stored - -Your configuration, instruction libraries (`.ins`), and reconstructions (`.stn`) live under your documents folder, in `SampleToNES/`: - -- Windows: `C:\Users\\Documents\SampleToNES` -- Linux: `/home//Documents/SampleToNES` -- macOS: `/Users//Documents/SampleToNES` - -### Command line - -Every operation is a named command, and `sampletones` alone starts the interface: - -```sh -sampletones run --config # start with a custom config -sampletones open # start with a project, reconstruction or library loaded -sampletones convert --config -o # reconstruct a recording without the GUI -sampletones library --config # generate an instruction library -``` +`sampletones` starts the app. Your work is saved in a `SampleToNES` folder inside your documents +folder. -Run `sampletones --help` for the commands and `sampletones --help` for a command's options. The [command-line guide](https://github.com/JakimPL/SampleToNES/blob/main/docs/guide/command-line.md) explains them. +Every operation is also a named command — `sampletones convert ` reconstructs a +recording without the interface, for example. Run `sampletones --help` for the list, and see the +[command-line guide](https://github.com/JakimPL/SampleToNES/blob/main/docs/guide/command-line.md). ## Documentation diff --git a/docs/api/index.md b/docs/api/index.md index 2c144f8f9..03961e649 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -5,29 +5,10 @@ This page is for using _SampleToNES_ as a library in your own Python code. Use i Names come from two packages: - The names in the table below come from `sampletones`: `from sampletones import ...`. -- The examples also use a few helpers from `sampletones_core`: `write_wave`, `ensure_library`, `DEFAULT_CHANNELS`, and the FamiTracker instrument writers. Import them with the full path each example shows. +- The examples also use a few helpers from `sampletones_core` that are not in the table: `write_wave`, `ensure_library`, `DEFAULT_CHANNELS`, and the FamiTracker instrument writers. Import them with the full path each example shows. ## Public surface -```python -from sampletones import ( - Config, - Window, - InstructionLibrary, - Reconstruction, - Reconstructor, - ChannelName, - Generator, - PulseGenerator, - TriangleGenerator, - NoiseGenerator, - Instruction, - PulseInstruction, - TriangleInstruction, - NoiseInstruction, -) -``` - | Name | Purpose | | --- | --- | | `Config` | generation configuration; build it with `Config.load(path)` or `Config.default()` | @@ -51,24 +32,18 @@ The package version is available as `sampletones.__version__`. from sampletones import Config, PulseGenerator, PulseInstruction from sampletones_core.audio.io import write_wave -# Load configuration config = Config.load("config.json") -# Prepare generator and instruction generator = PulseGenerator(config) instruction = PulseInstruction(on=True, pitch=55, volume=7, duty_cycle=2) - -# Generate waveform audio = generator(instruction) -# Save audio file -sample_rate = config.sample_rate -write_wave("pulse.wav", sample_rate, audio) +write_wave("pulse.wav", config.sample_rate, audio) ``` The output is a single `G2` square wave one frame long. -Each generator keeps an oscillator phase and clock. By default a call renders a standalone waveform and leaves that state where it was; pass `save=True` to advance it into the next call, so a sequence of instructions renders as one continuous signal: +Each generator keeps an oscillator phase and clock. A call renders a standalone waveform and leaves that state where it was. Pass `save=True` to advance the state into the next call, so a sequence of instructions renders as one continuous signal: ```python audio = generator(instruction, save=True) # advances the generator state @@ -86,7 +61,7 @@ config = Config.load("config.json") ensure_library(config) # builds the .ins library when it is missing or another version built it ``` -The same step is reached from the application's _Instructions_ tab, or on the command line with `sampletones library --config config.json`. +The application's _Instructions_ tab and `sampletones library --config config.json` do the same. ### Reconstruct a sample @@ -97,19 +72,14 @@ from sampletones import Config, Reconstructor from sampletones_core.audio.io import write_wave from sampletones_core.constants.enums import DEFAULT_CHANNELS -# Load configuration config = Config.load("config.json") -# Prepare the reconstructor for the channels the run may use +# The channels the run may use reconstructor = Reconstructor(config, frozenset(DEFAULT_CHANNELS)) -# Reconstruct an audio file and save the reconstruction reconstruction = reconstructor("sample.wav") reconstruction.save("reconstruction.stn") - -# Save the reconstruction waveform -sample_rate = config.sample_rate -write_wave("reconstruction.wav", sample_rate, reconstruction.approximation) +write_wave("reconstruction.wav", config.sample_rate, reconstruction.approximation) ``` ### Load a reconstruction @@ -139,4 +109,4 @@ for channel, features in reconstruction.export().items(): write_fti(f"{channel.value}.fti", instrument) ``` -This writes one `.fti` per channel, named after the channel. A complete FamiTracker `.ftm` module is assembled from a project in the application, not from a single reconstruction — see [FamiTracker formats](../formats/famitracker.md). +This writes one `.fti` per channel, named after the channel. A `.ftm` module comes from a project in the application. See [FamiTracker formats](../formats/famitracker.md). diff --git a/docs/concepts/compression.md b/docs/concepts/compression.md index 528e27efe..2a109c1e9 100644 --- a/docs/concepts/compression.md +++ b/docs/concepts/compression.md @@ -20,9 +20,9 @@ its sound registers are exactly the values the sequencer plays. A song reaches the console as **ticks** — the fixed-rate slices a reconstruction's envelopes advance through, the same slices the sequencer sounds a row in. On every -tick each of the four channels has a full set of register values, and written out -plainly that is 11 bytes a tick: three each for the two pulse channels and the -triangle, two for the noise. +tick each of the four [channels](../glossary.md#channel) has a full set of register +values, and written out plainly that is 11 bytes a tick: three each for the two pulse +channels and the triangle, two for the noise. At 60 ticks a second, 11 bytes a tick fills the space behind the driver in **49 seconds**. A song of three minutes needs 118800 bytes and the console has about @@ -49,11 +49,11 @@ The first thing the encoder does is unbraid them. Each register becomes a **plan one byte per tick, the whole song long — and each plane is a series of its own: a volume envelope that falls and holds, a pitch line that steps between notes, a duty cycle that barely moves. An idle channel's planes become one value repeated, which -costs almost nothing to state, and a plane holding zero throughout — a bend a channel never -makes — is left out of the block altogether. +costs almost nothing to state, and a plane standing at the value it is seeded to throughout is left out +of the block altogether: a bend a channel never makes, the control byte of a channel that never sounds. -This one change is most of the win. Split into planes and coded, the three-minute -arrangement falls from 11 bytes a tick to about 1.7. +This one change is most of the win. Split into planes and coded, the three-minute arrangement falls from +11 bytes a tick to 3.5. ### 2.2 Pitches instead of dividers @@ -63,10 +63,10 @@ dividers, and the steps between them are uneven. The encoder replaces the two divider planes with a **pitch index** — how far the frame's note sits above the lowest pitch the table covers — and a **bend**, the divider steps the tick stands away from that note's own divider. The song block carries a table the driver resolves the index through -and adds the bend to. A tone channel is therefore three planes, the same count as the -registers it writes. Counting a bend from the note keeps it the same bytes wherever a row -transposes the note to; only a bend past the signed byte is counted from the pitch lying -nearest the divider instead. +and adds the bend to. A pulse channel is therefore three planes. The triangle is two: it sounds at one +level, so its value plane carries the pitch and names silence with an index no pitch uses. Counting a bend +from the note keeps it the same bytes wherever a row transposes the note to. Only a bend past the signed +byte is counted from the pitch lying nearest the divider instead. The bend plane holds a value only where a note bends. The value plane's top bit flags each bent note from its first bent tick to its last, and the bend plane holds those ticks' steps @@ -74,32 +74,49 @@ alone, so it runs on a clock of its own: a note played straight costs it nothing channel that never bends leaves it out of the block. A loop re-enters it at the value its flags have reached, which is a boundary like any other. -Before any phrase, trading two dividers for an index and a bend is worth little: on the -arrangement §6 measures, which bends most of its notes, it costs about 2 %. What it earns -is that **a pitch index can be transposed and a divider cannot.** The same figure played +Before any phrase, trading two dividers for an index and a bend already pays. On the arrangement §6 +measures, which bends most of its notes, the planes take 3.210 bytes a tick against 3.517, because a bend +plane holds a value only where a note bends. What it earns beyond that is that **a pitch index can be +transposed and a divider cannot.** The same figure played at five pitches is five unrelated byte sequences in divider space; in index space it is one sequence and five offsets, its bend the same bytes throughout. That is what turns a repeated sample into a single dictionary entry in §5. +### 2.3 A value and the ticks it lasts + +A register rarely reads every bit of the byte written to it. A pulse control byte holds a duty cycle and a +volume around two bits the hardware wants set. A noise control byte holds a volume under the same two. A +noise period byte holds a mode bit above three the register reads nothing from. Those spare bits carry no +sound, and a long reconstruction spends most of its planes repeating one value. + +So a plane states the value and the ticks it lasts in the same byte: the value in the bits the register +reads, the count in the bits it ignores. That byte is a **symbol**, and a run costs one byte however long +it lasts, up to the count the plane's own spare bits reach. The driver masks a symbol to the register byte +and ors in the bits the hardware fixes, and counting one repeat off is a subtraction. + +A plane whose register reads all eight bits keeps a symbol a tick, so both kinds read the same way and the +block states no division of its own. **Every count in the token language counts symbols**, and a symbol +covers the ticks its own count states. + ## 3. The token language -Each plane is written as a sequence of **tokens**, and the driver reads them forward, -one tick at a time. There are three things a token can say, and the opcode byte's top +Each plane is written as a sequence of **tokens**, and the driver reads them forward as +the song plays. There are three things a token can say, and the opcode byte's top two bits say which: -- **Hold** — keep the value the plane reached, for up to 64 ticks. One byte. -- **Literal** — take the next few bytes, one per tick, up to 64 of them. One byte plus +- **Hold** — keep the value the plane reached, for up to 64 symbols. One byte. +- **Literal** — take the next few bytes, one symbol each, up to 64 of them. One byte plus the values. -- **Phrase** — play entry *p* of the dictionary, for up to 256 ticks, optionally with +- **Phrase** — play entry *p* of the dictionary, for up to 256 symbols, optionally with every value shifted. Two bytes, three if the phrase needs a full id byte or a shift, four if it needs both. Literals alone can write any plane, so the codec always has an answer; holds and phrases are what make that answer short. -Two properties of the encoding do most of the work later: +Three properties of the encoding do most of the work later: -**A token's count is a duration, not a length.** A phrase token says how many *ticks* +**A token's count is a duration, not a length.** A phrase token says how many *symbols* it covers, and that may run past the phrase's last value — past the end, the plane holds that value onward. This is what a note does when its envelope has finished and the note is still sounding, and it means one dictionary entry serves the same figure @@ -112,6 +129,11 @@ added to each of its values, wrapping at 256. On the 6502 that is a single addit and a fall in pitch is simply the byte that wraps around to it. Together with the duration rule, one entry covers every pitch *and* every length a figure is played at. +**A phrase may state its own count.** A figure played at one length throughout states that length once, in +the phrase's own entry, and every token playing it there names the phrase alone. The bit that says so +comes out of the opcode's id field, so a phrase opcode names 31 ids outright where a hold counts to 64. +The phrases a song leans on hardest take those ids, and the rest name themselves in a further byte. + ## 4. Reading a plane the cheapest way A plane usually admits many readings. A run of eight identical values can be one hold, @@ -120,21 +142,21 @@ readings cost different numbers of bytes, and the differences compound over a so thousands of ticks. So the encoder does not pick a reading by rules of thumb; it searches for the cheapest -one. The plane becomes a graph: each tick is a node, each token that could start there -is an edge to the tick after the ones it covers, and the edge's weight is the bytes +one. The plane becomes a graph: each symbol is a node, each token that could start there +is an edge to the symbol after the ones it covers, and the edge's weight is the bytes that token takes. **The cheapest path across the plane is its encoding** — and because the weights are bytes, the search optimizes the very quantity that has to fit in the program area. ### 4.1 The edges -From each tick, the encoder offers: +From each symbol, the encoder offers: - a **hold**, where the value repeats the one before it, running as far as the repeated - run does, capped at 64 ticks; -- a **literal**, reaching this tick from the cheapest start within the last 64 ticks; + run does, capped at 64 symbols; +- a **literal**, reaching this symbol from the cheapest start within the last 64 symbols; - a **phrase**, one edge per dictionary entry the plane plays from here, covering as - many ticks as the plane agrees with it plus however long its last value carries. + many symbols as the plane agrees with it plus however long its last value carries. Literals need care, because every one of the 64 possible starts is a candidate and checking them all would make the parse quadratic. A literal costs its opcode and its @@ -144,10 +166,10 @@ whole plane's literals are priced in one pass. ### 4.2 Why the search beats taking the longest match -The obvious alternative — at each tick take the longest phrase that matches, otherwise +The obvious alternative — at each symbol take the longest phrase that matches, otherwise hold, otherwise spell out — is wrong in a way that shows up constantly. Taking a -40-tick phrase for two bytes looks better than taking a 30-tick one, until it turns out -that stopping at 30 would have let the next 200 ticks be a single hold. Costs also +40-symbol phrase for two bytes looks better than taking a 30-symbol one, until it turns out +that stopping at 30 would have let the next 200 symbols be a single hold. Costs also depend on the dictionary: the same phrase is two bytes with a cheap id and four with an escaped id and a shift. The search weighs those against each other; a rule of thumb cannot. @@ -157,8 +179,8 @@ cannot. A song that repeats re-enters its streams partway through rather than at the beginning, and the driver arrives there with nothing behind it: it points each plane at a byte the header names and starts reading. For that to work, the loop tick has to **begin** a -token on every plane, and that token has to state its values outright rather than lean -on a value the plane reached earlier. +symbol, and a token, on every plane, and that token has to state its values outright rather +than lean on a value the plane reached earlier. The parse takes this as a constraint. The loop tick is a boundary: tokens may end there and start there, and none may span it. A hold is barred from starting there, since a @@ -172,7 +194,7 @@ runs of values a plane plays — stored once in the song block and named by toke wherever they occur. A phrase's position in the table is its id, and the ids are not equally priced: the -first 63 ride inside the opcode byte, and the rest need a byte of their own. So the +first 31 ride inside the opcode byte, and the rest need a byte of their own. So the order of the table is part of the encoding, and the phrases a song leans on hardest belong at the front. @@ -183,8 +205,8 @@ A song is built by placing samples at rows, so the shapes its planes repeat are planes it writes, at the pitch and level it was reconstructed at, and every row playing that sample becomes a token naming those entries with the shift the row asks for. No search is involved, and this is where most of a project's compression comes from: on -the three-minute arrangement the instruments alone take it from 1.6 bytes a tick to -1.13, and transposition to 0.98. +the three-minute arrangement §6 measures, the instruments alone take it from 2.980 bytes +a tick to 1.717, and transposition to 1.447. A project can offer more phrases than the table holds. When it does, each is weighed by what it would actually spare the song, and the ones that pay most keep their place; the @@ -198,7 +220,7 @@ transitions between rows, and the whole of a reconstruction export, where each s played exactly once and nothing repeats by construction. The search works over that residue — the spans the current parse still spells out as -literals. It gathers every run of 3 to 48 ticks that appears in them and groups them by +literals. It gathers every run of 3 to 48 symbols that appears in them and groups them by **shape**: the step from each value to the next, so a figure played at five pitches collects into one candidate seen five times. Candidates occurring at least twice, in places that do not overlap, are scored: @@ -233,17 +255,17 @@ is parsed again, and the process repeats until it settles — a few rounds at mo ### 5.4 Matching is measured once The encoder parses the whole song many times: once as a baseline, once per confirmed -candidate, once per settling round. A parse asks the same question at every tick — what -does this phrase play here, and for how many ticks — and the answer depends only on the +candidate, once per settling round. A parse asks the same question at every symbol — what +does this phrase play here, and for how many symbols — and the answer depends only on the plane and the phrase. It cannot change between parses. So it is measured once per plane per phrase and kept for the whole encoding. A search round that adds one phrase measures that one phrase; everything already in the table answers from the reading taken when it arrived. This turns the cost of an encode from *parses × dictionary* into *dictionary*, and it is the largest reason a three-minute -song encodes in about two seconds. Phrases are also offered only at the ticks whose +song encodes in about two seconds. Phrases are also offered only at the symbols whose first two steps match their own, so a reading covers the handful of places a phrase -could begin rather than every tick of the song. +could begin rather than every symbol of the song. ## 6. What it achieves @@ -252,26 +274,28 @@ above it: | what is stored | bytes per tick | ratio | ticks that fit | |---|---|---|---| -| a record per tick per channel | 11.000 | 1.00 | 2914 | -| planes, coded | 3.517 | 3.13 | 9115 | -| planes with a pitch index and a bend | 3.604 | 3.05 | 8885 | -| phrases from the instruments | 1.989 | 5.53 | 16201 | -| phrases played transposed | 1.593 | 6.90 | 20294 | -| phrases from the search as well | **1.346** | **8.17** | **24130** | - -The arrangement bends most of the notes its pulse channel plays, and its bend plane -carries every one of them; a song played straight leaves its bend planes out of the -block. The whole song is 14534 bytes of the roughly 32000 available, and **24130 ticks -is 6.7 minutes at 60 Hz**, against the 49 seconds a record per tick reaches. Encoding it -costs between two and three seconds; decoding it costs the console around twenty -instructions per plane per tick, comfortably inside a video frame. - -`uv run sampletones codec report` writes this table over a corpus of songs, and the format's -constants are settled from it. Two of them were settled against expectation: splitting -the duty cycle out of the control byte into a plane of its own **costs** 9 %, because -volume and duty turn over together and a split pays two opcodes for what one covers; -and the pitch index earns its place through the transposition it makes possible, 20 %, -where directly it costs the bent arrangement about 2 %. +| a record per tick per channel | 11.000 | 1.00 | 2907 | +| planes, coded | 3.517 | 3.13 | 9092 | +| planes with a pitch index and a bend | 2.980 | 3.69 | 10731 | +| phrases from the instruments | 1.717 | 6.41 | 18750 | +| phrases played transposed | 1.447 | 7.60 | 22326 | +| phrases from the search as well | **0.874** | **12.59** | **37660** | + +The arrangement bends most of the notes its pulse channel plays, and its bend plane carries every one of +them. A song played straight leaves its bend planes out of the block. The whole song is 9435 bytes of the +roughly 32000 available, and **37660 ticks is 10.5 minutes at 60 Hz**, against the 49 seconds a record per +tick reaches. Encoding happens once, where the file is written. Decoding costs the console around twenty +instructions per plane per tick, and fewer on a tick a symbol still covers, comfortably inside a video +frame. + +The report writes this table over a corpus of songs, and the format's constants are settled from it. Two +of them were settled against expectation. Splitting the duty cycle out of the control byte into a plane of +its own **costs** bytes. On the arrangement above, encoded at every layer, it costs 16 % where no plane +packs, because volume and duty turn over together and a split pays two opcodes for what one covers, and +56 % where the planes pack as the format packs them, because a split plane also gives up the repeat count +its register's spare bits carry. The pitch index +earns its place through the transposition it makes possible, the fourth row against the fifth, and it pays +for itself directly as well (§2.2). An export chooses how far down these layers it goes. The **Level** it is written at names the layers read in order: *None* spells every plane out as literals, *Held sounds* adds holds, @@ -291,38 +315,13 @@ plays the same song. - **The dictionary holds 255 phrases.** A project of roughly 30 to 60 samples fills it from its instruments alone, at which point the search has no room left to work in and further samples compete for slots on measured value. -- **A phrase holds at most 255 values and a token covers at most 256 ticks.** Longer +- **A phrase holds at most 255 values and a token covers at most 256 symbols.** Longer figures are stated as several tokens, which costs a couple of bytes each time. - **Compression is per plane.** Two channels playing the same figure at once share dictionary entries, and nothing exploits the correlation between a channel's own control and value planes. -## Appendix — the shape of the format - -| quantity | value | -|---|---| -| planes | control and value for every channel, and a bend for each tone channel | -| bytes per tick before coding | 11 | -| ticks one hold covers | 1 to 64 | -| values one literal carries | 1 to 64 | -| ticks one phrase token covers | 1 to 256 | -| phrase ids inside the opcode | 63 | -| phrases in the dictionary | up to 255 | -| values in a phrase | up to 255 | -| candidate lengths the search gathers | 3 to 48 | -| decoder state on the console | 88 bytes of zero page, 8 per plane | - -Where things live: - -| concern | module | -|---|---| -| planes, and the pitch index | `sampletones_player.compression.planes`, `.pitch` | -| the token kinds and their byte costs | `sampletones_player.compression.tokens` | -| the cheapest reading of a plane | `sampletones_player.compression.parse` | -| the dictionary, its entries and its pruning | `sampletones_player.compression.dictionary` | -| phrases the instruments offer | `sampletones_player.compression.seeds` | -| phrases the search earns | `sampletones_player.compression.search` | -| how much work the search spends | `sampletones_player.compression.budget` | -| what a phrase plays against a plane | `sampletones_player.compression.matches` | -| encoding, and the decoder the driver is held to | `sampletones_player.compression.encode`, `.decode` | -| the opcode layout and its bounds | `sampletones_player.specification.compression` | +## Appendix — where the exact shape is written + +This page explains the scheme. The bytes themselves — the opcodes, the operand each carries and +the bounds they impose — are in [NSF export](../formats/nsf.md). diff --git a/docs/concepts/instruction-library.md b/docs/concepts/instruction-library.md index d2fedd0b3..fca6c8da6 100644 --- a/docs/concepts/instruction-library.md +++ b/docs/concepts/instruction-library.md @@ -1,70 +1,55 @@ # Instruction library -An instruction library is the catalog of NES sounds that _SampleToNES_ -searches when it reconstructs audio. It holds every instruction a channel can -play — each combination of pitch, volume, timbre and on/off state — together -with the waveform that instruction produces and a description of its frequency -content. Reconstruction is then a matter of searching this catalog: for each -slice of the input, the engine looks for the library entries whose combined -sound is closest to that slice. [Reconstruction algorithms](reconstruction.md) -describes that search; this page describes the catalog it searches. +This page defines what an instruction library is and what keys it. Read it before changing how a library +is generated or how its key is built. [Reconstruction algorithms](reconstruction.md) describes the search +over the catalog. [Instruction libraries](../formats/instruction-libraries.md) documents the file. -## Why the library is precomputed +An instruction library is the catalog of NES sounds _SampleToNES_ searches when it reconstructs audio. It +has every instruction a channel can play: each combination of pitch, volume, timbre and on/off state. Each +entry has the waveform its instruction produces and a description of its frequency content. +Reconstruction searches this catalog. For each slice of the input, the engine looks for the entries whose +combined sound is closest to that slice. -The number of distinct instructions is large but fixed — a few thousand per -channel — and the same candidates are compared against every frame of every -sample. Rendering each candidate's waveform and analyzing its spectrum once, up -front, turns the per-frame work into a lookup instead of a re-synthesis. A -library is therefore built once for a given configuration and reused across -every reconstruction that shares it. +## Why the library is precomputed -The library is also a first-class artifact in its own right: you can generate, -browse and audition one from the _Instructions_ tab without reconstructing -anything, which is a good way to hear what a given channel and configuration can -actually produce. +The number of distinct instructions is large but fixed, and the same candidates are compared against every +frame of every sample. The library renders each candidate's waveform and analyzes its spectrum once, up +front. That turns the per-frame work into a lookup instead of a re-synthesis. A library is built once for +a given configuration and reused across every reconstruction that shares it. ## Phase independence -Two recordings of the same note can look completely different sample-by-sample -depending on where in its cycle each one starts — their phase. To keep matching -about *what a sound is* rather than *when it happened to begin*, each candidate's -stored spectrum is computed as an average over many phase offsets. The result is -essentially phase-independent, so a candidate matches a frame on the shape of its -spectrum, not on an accident of alignment. +Two recordings of the same note can look completely different sample by sample, depending on where in its +cycle each one starts. That position is the phase. Each candidate's stored spectrum is an average over +many phase offsets, so a candidate matches a frame on the shape of its spectrum and not on where its cycle +begins. [The candidate catalog](reconstruction.md#31-the-candidate-catalog-library) describes how the +average is made. ## What a library is keyed by -A library depends on the configuration values that change the rendered waveforms -or the way their spectra are measured: - -* the **sample rate** and **NES frequency**, which together set the length of a - frame and so the length of each rendered waveform; -* the **spectrum method** (`fft`, `logfft` or `cqt`) and **transformation gamma**, - which set how each waveform's frequency content is measured and weighted. +A library depends on the configuration values that change the rendered waveforms or the way their spectra +are measured: -These values form the library's key. Changing any of them describes a different -sound space, so it selects a different library — and generates a fresh one if -none exists yet. What the spectrum method and gamma actually do is covered in -[Reconstruction algorithms](reconstruction.md) (§3.2–3.3), because the target -audio is measured the same way; the library and the target always share one -representation so their spectra are directly comparable. +* the **sample rate** and **NES frequency**, which together set the length of a frame and so the length + of each rendered waveform; +* the **spectrum method** (`fft`, `logfft` or `cqt`) and **transformation gamma**, which set how each + waveform's frequency content is measured and weighted. -## Generating and exploring +These values form the library's key. Changing any of them describes a different sound space, so it +selects a different library and generates a fresh one if none exists. The target audio is measured the +same way, so the library and the target always share one representation and their spectra are directly +comparable. [Spectrum methods](reconstruction.md#32-spectrum-methods-fft-log-fft-and-cqt) explains what the +spectrum method and gamma do. -Generate a library from the _Instructions_ tab, or on the command line with -`sampletones library`. Generation renders every instruction and stores its -waveform and spectrum; a configuration that has no library yet is also built -automatically the first time a reconstruction needs it. Regenerating a library -that already exists replaces it. +## Versions and rebuilding -A library belongs to the version of _SampleToNES_ that built it. A library built -by another version is rebuilt in its place the first time a reconstruction needs -it, the same way a missing one is built. +A library belongs to the version of _SampleToNES_ that built it. A library built by another version is +rebuilt in its place the first time a reconstruction needs it. A configuration with no library is built +the same way. Regenerating a library that already exists replaces it. +[Data compatibility](../development/release/compatibility.md) describes the version rule. -The _Instructions_ tab lists the instructions in a library and shows the selected -one's waveform and spectrum alongside a player, so a library doubles as a way to -explore the raw material a reconstruction is assembled from. +Building a library by hand is covered in the guide, under +[building a library yourself](../guide/converting.md#building-a-library-yourself). -On disk a library is a single `.ins` file whose name encodes its configuration -key; see [Instruction libraries](../formats/instruction-libraries.md) for the -file format. +On disk a library is a single `.ins` file whose name encodes its configuration key. See +[Instruction libraries](../formats/instruction-libraries.md) for the file format. diff --git a/docs/concepts/project.md b/docs/concepts/project.md index 3a403a66b..e654584b3 100644 --- a/docs/concepts/project.md +++ b/docs/concepts/project.md @@ -1,35 +1,33 @@ # Project -A project is a whole composition in _SampleToNES_: a song written for the four NES -channels, together with the voices it is built from. Where a -[reconstruction](reconstruction.md) is a single converted sound, a project holds -many of them, the instruments written by hand beside them, and the arrangement that -plays them all, so an entire piece lives as one file. +This page says what a project is and what it holds. Read it before changing what a saved project +contains. [Projects](../formats/projects.md) documents the file. + +A project is a whole composition in _SampleToNES_: a song written for the four NES channels, with the +voices it is built from. A [reconstruction](reconstruction.md) is a single converted sound. A project has +many of them, the instruments written by hand beside them, and the arrangement that plays them all. An +entire piece lives in one file. ## What a project brings together -- the **voices** — the reconstructions you have imported and the instruments you have - written by hand, each of them something a row can play; -- the **song** — the arrangement itself: the patterns written for each channel and - the order they play in; -- the **timing and details** — the tempo, speed, and NES frequency the song plays - at, and the title, author, and comment that describe it. +- The [**voices**](../glossary.md#voice): the reconstructions you have imported and the instruments you + have written by hand. A [row](../glossary.md#row) can play any of them. +- The **song**: the [patterns](../glossary.md#pattern) written for each channel and the order they play + in. +- The **settings and info**: the tempo, speed and NES frequency the song plays at, and the title, author + and comment that describe it. -You create and edit all of this on the [Sequencer](../guide/sequencer.md) tab, and -export the finished piece as a FamiTracker [module](../formats/famitracker.md). +The [sequencer guide](../guide/sequencer.md) covers writing a project and exporting it. ## Self-contained and portable -A project embeds the reconstructions it uses rather than pointing at them elsewhere -on disk, and writes each hand-written instrument into the document itself, so moving or sharing the -file carries the whole composition — the arrangement and every sound it needs. The -embedded reconstructions are -[detached](../formats/reconstructions.md#detached-reconstructions) from their -source-audio paths, which mean nothing on another machine, so the project opens the -same wherever it goes. +A project stores the reconstructions it uses inside its own file. It also stores each hand-written +instrument there. Moving or sharing the file therefore moves the whole composition: the arrangement and +every sound it needs. The stored reconstructions are +[detached](../formats/reconstructions.md#detached-reconstructions) from their source-audio paths, because +those paths mean nothing on another machine. The project opens the same wherever it goes. ## On disk -A project is saved as a `.stp` file — a small document describing the song and its -settings, alongside the embedded reconstructions. -[Projects](../formats/projects.md) documents that structure. +A project is saved as a `.stp` file: a document describing the song and its settings, with the embedded +reconstructions beside it. [Projects](../formats/projects.md) documents that structure. diff --git a/docs/concepts/reconstruction.md b/docs/concepts/reconstruction.md index 22ff9124d..5cd769714 100644 --- a/docs/concepts/reconstruction.md +++ b/docs/concepts/reconstruction.md @@ -1,103 +1,96 @@ # Reconstruction algorithms -This document explains how _SampleToNES_ turns an arbitrary audio sample into a -*reconstruction* — a sequence of NES instructions that, when played back on the -console's sound hardware, approximates the original. It is written to be readable -without prior knowledge of the codebase, while pointing at the packages that -implement each part. +This document explains how _SampleToNES_ turns an audio sample into a *reconstruction*: a sequence of NES +instructions that approximates the original when it plays on the console's sound hardware. Read it to see +how a frame's sound is described, scored and chosen, and where each setting acts. You can read it without +reading the source code. -The tunable choices described here are set empirically; the experiment that -picks them is described in [Calibration](../tools/calibration.md). +The tunable choices described here are set empirically. [Calibration](../tools/calibration.md) describes +the experiment that sets them. ## 1. The problem -The NES sound chip (Ricoh 2A03 APU) can only produce a few simple, fixed -waveforms across four usable channels: - -- two **pulse** (square) channels — each with 4 duty cycles and 15 volume levels; -- one **triangle** channel — fixed shape and amplitude, pitch only; -- one **noise** channel — a pseudo-random LFSR generator with 16 periods, 15 - volume levels and a short/long mode. - -A program steers these channels by issuing *instructions* a few dozen times per -second (for example, *pulse 1: note A-4, volume 12, 50 % duty*). Approximating an -arbitrary sound this way produces a **reconstruction**: one instruction stream per -channel whose mixed, rendered output resembles the input as closely as the -hardware permits. By default a reconstruction uses one pulse channel, the triangle -and the noise; the second pulse can be enabled in the configuration. The channel -models live in `sampletones_core.generators` and the instruction value types in -`sampletones_core.instructions`. - -Reconstruction is a **search problem**. The input is cut into short, fixed-length -frames, and within each frame at most one instruction per channel is in effect. For -every frame the system must pick, from a large but finite catalog of NES -waveforms, the combination of instructions whose mixed output best matches that +The NES sound chip ([Ricoh 2A03 APU](../glossary.md#2a03-apu)) produces a few simple, fixed waveforms +across four usable channels: + +- two [**pulse**](../glossary.md#pulse-square) (square) channels, each with 4 + [duty cycles](../glossary.md#duty-cycle) and 15 volume levels; +- one [**triangle**](../glossary.md#triangle) channel of fixed shape and amplitude, with pitch only; +- one [**noise**](../glossary.md#noise) channel: a pseudo-random [LFSR](../glossary.md#lfsr) generator + with 16 periods, 15 volume levels and a short/long mode. + +The noise period setting divides the APU clock into the LFSR's shift rate, +`APU_CLOCK / NOISE_PERIODS[index]`. That rate runs from 440.0 Hz at index 0 to 447443.2 Hz at index 15. In +short mode the register repeats every 93 shifts, so index 15 sounds as a tone at about 4811 Hz. Short +mode's output bit is set 17.2% of the time, against 50% in long mode, and that imbalance gives it a +metallic timbre. + +A program steers these channels by issuing *instructions* at the +[NES frequency](../glossary.md#nes-frequency), for example *pulse 1: note A-4, volume 12, 50 % duty*. +Approximating an arbitrary sound this way produces a **reconstruction**: one instruction stream per +channel whose mixed, rendered output resembles the input as closely as the hardware permits. By default a +reconstruction uses one pulse channel, the triangle and the noise. The second pulse can be switched on +when a recording's channels are chosen. + +Reconstruction is a **search problem**. The input is cut into short, fixed-length frames, and within each +frame at most one instruction per channel is in effect. For every frame the system must pick, from a large +but finite catalog of NES waveforms, the combination of instructions whose mixed output best matches that slice of audio. Two ingredients define the system: -- a **criterion** that scores how well a candidate matches the target (§4), and -- a **selection strategy** that searches the catalog efficiently (§5). +- a **criterion** that scores how well a candidate matches the target + ([section 4](#4-scoring-a-candidate-the-criterion)), and +- a **selection strategy** that searches the catalog efficiently ([section 5](#5-choosing-instructions)). -Everything is compared in a perceptually-weighted **frequency** representation -rather than raw samples, because two sounds that are perceptually identical can -have very different waveforms depending on phase. +Everything is compared in a perceptually weighted **frequency** representation instead of raw samples, +because two sounds that are perceptually identical can have very different waveforms depending on phase. ## 2. The pipeline -`Reconstructor` (`sampletones_core.reconstructions.reconstructor`) carries the -input through a fixed sequence of stages: - -1. **Load** the audio (`sampletones_core.audio`) — mix to mono, resample, and - optionally clean it up (normalize, quantize). Several sources load together, so - one scale drawn from the peak of their sum holds them at the balance they were - captured in. -2. **Set a working level** — scale the whole signal so its typical frame plays at the - level one channel renders at full volume, keeping quiet passages matchable (§3.4). -3. **Fragment** it into short, fixed-length frames - (`sampletones_core.fft.fragment`); from here on each channel holds one - instruction per frame. -4. **Describe each frame** by a spectral feature that captures its frequency - content (§3). -5. **Assign** every frame's channels to the sources, and with each channel the - candidates it may sound there, each source judged against its own audio by the - criterion (§5 and §4). -6. **Decode** each channel's stream, reading its candidates across the whole - recording (§5). -7. **Refine** each chosen note onto the divider the recording's own fundamental stands at (§6). -8. **Render** the chosen instructions back into audio through the generators, - keeping each oscillator continuous across frames. -9. **Reassemble** the channels into the final approximation and package it, with the - instruction streams, as a `Reconstruction`. - -Stages 3–7 are where the algorithms described below live; the rest is preparation -and playback. - -A run says which of these it is in as it passes through them, so a reader watching a -conversion sees it move rather than waiting for the file. `ReconstructionStage` gathers -the eight steps into the four a reader is told apart — loading, matching, decoding, -rendering — and states the share each holds of the whole run; -[`progress.md`](../development/progress.md) describes how that account reaches the -screen from the worker process it is made in. +A reconstruction runs through a fixed sequence of stages: + +1. **Load** the audio: mix to mono, resample, and optionally clean it up (normalize, quantize). Several + sources load together, so one scale drawn from the peak of their sum keeps them at the balance they + were captured in. +2. **Set a working level.** Scale the whole signal so its typical frame plays at the level one channel + renders at full volume, which keeps quiet passages matchable + ([section 3.4](#34-the-working-level-coefficient)). +3. **Fragment** the signal into short, fixed-length frames. From here on each channel has one instruction + per frame. +4. **Describe each frame** by a spectral feature that captures its frequency content + ([section 3](#3-representing-a-frame)). +5. **Assign** every frame's channels to the sources. Each channel also gets the candidates it may sound + there, and each source is judged against its own audio by the criterion + ([section 5](#5-choosing-instructions) and [section 4](#4-scoring-a-candidate-the-criterion)). +6. **Decode** each channel's stream, reading its candidates across the whole recording + ([section 5](#5-choosing-instructions)). +7. **Refine** each chosen note onto the [divider](../glossary.md#divider) the recording's own fundamental + stands at ([section 6](#6-refining-the-pitch)). +8. **Render** the chosen instructions back into audio through the generators, keeping each oscillator + continuous across frames. +9. **Reassemble** the channels into the final approximation, and save it with the instruction streams as + a reconstruction. + +Stages 3–7 are where the algorithms described below live. The rest is preparation and playback. + +While it runs, a conversion shows four stages: loading, matching, decoding and gathering. ## 3. Representing a frame ### 3.1 The candidate catalog (library) -Before any reconstruction, `sampletones_core.library` precomputes a **library**: for -every possible instruction it renders the waveform its generator produces and stores -the corresponding spectral feature. Because NES waveforms are periodic, a -candidate's feature is computed from its power spectrum averaged over many phase -offsets, which makes it essentially phase-independent — matching then compares -spectral *shape* rather than an accident of alignment. The library is keyed by the -parameters that affect it (sample rate, frame size, spectrum method, gamma, …) so a -configuration change produces a fresh library. See -[Instruction library](instruction-library.md) for the library as an artifact — how it -is generated, explored and keyed. +Before any reconstruction, the program precomputes a **library**. For every possible instruction it +renders the waveform that instruction produces and stores the corresponding spectral feature. NES +waveforms are periodic, so a candidate's feature is computed from its power spectrum averaged over many +phase offsets. That makes it essentially phase-independent, and matching compares spectral *shape* and +not an accident of alignment. The library is keyed by the parameters that affect it (sample rate, frame +size, spectrum method, gamma, …), so a configuration change produces a fresh library. See +[Instruction library](instruction-library.md) for how it is generated and keyed. ### 3.2 Spectrum methods: FFT, log-FFT and CQT -The feature of a frame is a frequency *histogram* (`sampletones_core.structures`), -produced by one of three methods (`sampletones_core.fft.spectrum`, -configurable via `library.spectrum_method`): +The feature of a frame is a frequency *histogram*, produced by one of three methods +(`library.spectrum_method`). The figures below are for 44.1 kHz audio at a 60 Hz change +rate: | method | frequency axis | resolution | time support | |----------|---------------------------|-----------------------------------------|---------------------------| @@ -123,9 +116,9 @@ sharper frequency resolution requires a longer time window, and vice versa): It is the default. The price is time support: its low-frequency basis functions are long (hundreds of milliseconds), so brief events are smeared in time at the low end. _SampleToNES_ - computes the CQT **once over the whole signal** with a hop of one frame - (`calculate_cqt_spectrum_columns`), so each frame's energy is reported at its own - time position and the per-frame columns line up with the FFT path's frame centers. + computes the CQT **once over the whole signal** with a hop of one frame, so each + frame's energy is reported at its own time position and the per-frame columns line up + with the FFT path's frame centers. The target and the library candidates are always described by the *same* method, so their features are directly comparable bin by bin. All three methods share one scale @@ -136,7 +129,7 @@ by its energy gain). ### 3.3 The gamma transform Whatever the method, the raw power spectrum is mapped into a "feature space" by a -Yeo-Johnson-family transform (`sampletones_core.fft.transformer`) controlled by a +Yeo-Johnson-family transform controlled by a `transformation_gamma` in `[0, 100]`: - `gamma = 0` → identity: the feature is the power spectrum (the default); @@ -152,305 +145,267 @@ at every gamma. ### 3.4 The working level (coefficient) -A single **coefficient** scales the input before matching so that its typical frame -plays at the level one channel renders at full volume. The typical frame is a -*robust* level — a high percentile of the per-frame RMS levels over the audible frames -(`active_frame_level` in `sampletones_core.audio`) — so a lone transient (a kick, a -click) saturates to the loudest available note while the bulk of the signal stays -within reach of the quietest one. RMS measures how much sound a frame carries, which -is what a channel's volume renders, whatever the waveform's crest. +A single **coefficient** scales the input before matching, so its typical frame plays at the level one +channel renders at full volume. The typical frame is a *robust* level: a high percentile of the per-frame +RMS levels over the audible frames. A lone transient such as a kick or a click therefore saturates to the +loudest available note, while the bulk of the signal stays within reach of the quietest one. RMS measures +how much sound a frame carries. A channel's volume renders that quantity, whatever the waveform's crest. -That level is brought to the full-scale RMS level of the quietest tone channel the -setup covers — the triangle, whose RMS level is its peak over √3; a pulse, which swings -between two levels, when no triangle is covered; the noise channel for a noise-only -setup. A steady tone then lands at what a single tone channel renders whole, so one -channel can answer it, and louder frames call on more channels. +That level is brought to the full-scale RMS level of the quietest tone channel the setup covers. This is +the triangle, whose RMS level is its peak over √3. A pulse, which swings between two levels, takes over +when no triangle is covered, and the noise channel does for a noise-only setup. A steady tone then lands at +what a single tone channel renders whole, so one channel can answer it, and louder frames call on more +channels. ## 4. Scoring a candidate: the criterion -`Criterion` (`sampletones_core.reconstructions.criterion`) scores a candidate -against the target frame as a weighted sum of a spectral and a temporal term: +The **criterion** scores a candidate against the target frame as a weighted sum of a spectral and a +temporal term: ``` -cost = α · spectral + β · temporal (default α = 0.8, β = 0.2) +cost = α · spectral + β · temporal ``` -- **spectral** compares the two frequency features with a perceptually-weighted - distance, normalized by the target's own energy so the score is about *shape*. The - per-bin distance is configurable — squared error, absolute error, or a **β-divergence** - (the default: a Kullback–Leibler-style measure that penalizes leaving target energy - uncovered more strongly than adding energy beyond it). Both sides are measured - above a **floor the configured dynamic range sets under the frame's loudest bin** - (`generation.metric.dynamic_range_decibels`, 60 dB as shipped), so an addition costs - what it adds wherever it stays audible beside what the frame sounds — quiet noise - under a loud tone — and a frame of noise costs what a channel leaves out of it. A - frame quieter than `generation.metric.silence_floor` is measured from that level, - which keeps a silent frame's score finite. - Bins are weighted by their span in auditory critical bands (the ERB scale) times the - K-weighting loudness curve (ITU-R BS.1770), so each bin counts in proportion to the - hearing resolution and loudness contribution it represents. -- **temporal** measures the target *waveform* against what the frame's channels are - expected to render, normalized by the target frame's own level so the - spectral/temporal blend holds across frame loudness. A candidate whose frames repeat - one waveform shape — a note, or noise whose register cycle fits inside a frame — - renders its waveform at its best phase against what the other channels leave of the - target, which makes the term measure waveform *shape*, a property the magnitude - spectrum discards. A candidate whose frames show different stretches of a - pseudo-random sequence renders its mean level with a spread about it: for waveforms - expected to sum to `E` with a per-sample variance `V`, +α and β are the `spectral_loss_weight` and `temporal_loss_weight` settings. + +- **spectral** compares the two frequency features with a perceptually weighted distance. The distance is + normalized by the target's own energy, so the score is about *shape*. The per-bin distance is + configurable: squared error, absolute error, or a **β-divergence**, which is the default. A + β-divergence is a Kullback–Leibler-style measure that penalizes leaving target energy uncovered more + strongly than adding energy beyond it. + + Both sides are measured above a floor set under the frame's loudest bin by the configured dynamic range + (`generation.metric.dynamic_range_decibels`). An addition therefore costs what it adds wherever it stays + audible beside what the frame sounds, such as quiet noise under a loud tone. A frame of noise costs what + a channel leaves out of it. A frame quieter than `generation.metric.silence_floor` is measured from that + level, which keeps a silent frame's score finite. + + Bins are weighted by their span in auditory critical bands (the ERB scale) times the K-weighting + loudness curve (ITU-R BS.1770). Each bin then counts in proportion to the hearing resolution and + loudness contribution it represents. +- **temporal** measures the target *waveform* against what the frame's channels are expected to render. It + is normalized by the target frame's own level, so the spectral/temporal blend holds across frame + loudness. A candidate whose frames repeat one waveform shape (a note, or noise whose register cycle fits + inside a frame) renders its waveform at its best phase against what the other channels leave of the + target. That makes the term measure waveform *shape*, a property the magnitude spectrum discards. A + candidate whose frames show different stretches of a pseudo-random sequence renders its mean level with + a spread about it. For waveforms expected to sum to `E` with a per-sample variance `V`, `E mean((t − x)²) = mean((t − E)²) + V`, whose root normalizes as above. -A lower cost is a better match. The criterion evaluates many candidates at once and, -on machines with a GPU, runs on the array backend in `sampletones_shared`. +A lower cost is a better match. The criterion scores many candidates at once, and runs on the graphics +card where the machine has one. ## 5. Choosing instructions -Two questions settle what a frame plays, and each has its own owner. **Ownership** — -which channel a source holds this frame — is answered by the assignment in -`sampletones_core.reconstructions.reconstructor.stems.assignment`. **The stream** — -what a channel plays across the frames it holds — is answered by a decoder in -`sampletones_core.reconstructions.reconstructor.decoder`, named by -`generation.decoder.selector`. Both work from the same candidate scoring -(`reconstructor/matching.py`), the same criterion and the same library. - -The assignment leaves every channel in play a **column** per frame: the candidates -that channel may sound there, best first. The decoder reads those columns into one -candidate per frame. Each decoder states how wide a column it reads, and the -assignment builds columns to exactly that width. - -A candidate is scored by the **frame's cost with it sounding** beside the picks its source -already holds in that frame, the channel's silence among the candidates. The picks add up -to a **mix**: their phase-averaged power spectra add, and so do the waveforms they are -expected to render and their variances. Scoring runs in two stages: every candidate of -one channel's kind is added to the mix and ranked by the phase-independent spectral term, -and the best `top_k` together with the kind's silence are then re-scored with the full -criterion. A candidate whose frames repeat one shape adds its waveform aligned to what -the mix leaves of the target when `find_best_phase` is on, and its library sample from -the start otherwise; either phase stands in for the one the generator reaches when the -frame is rendered. A candidate whose frames show different stretches of a sequence -renders whatever stretch its channel has reached, so it adds its mean level and its -variance whatever `find_best_phase` says. The scored candidates, best first and silence -ahead of an equal cost, form the channel's column; `top_k` sets how many of them a wide -decoder reads. +Two questions settle what a frame plays. Each is answered separately. + +- **Ownership**: which channels a source holds in this frame. The assignment + ([section 5.1](#51-assigning-channels)) answers it. +- **The stream**: what a channel plays across the frames it holds. A decoder answers it, and the + `generation.decoder.selector` setting chooses the decoder. + +Both work from the same candidate scoring, the same criterion and the same library. + +A **source** is one recording in the conversion, a [stem](../glossary.md#stem). A conversion from a single +file has one source. Three more terms are used below. A [pick](../glossary.md#pick) gives one channel to +one source in a frame, with the candidate it sounds. A [mix](../glossary.md#mix) is the combined sound of +a source's picks in a frame. A [column](../glossary.md#column) is the candidates one channel may sound in a +frame, best first. + +The assignment leaves every channel in play a column per frame. The decoder reads those columns into one +candidate per frame. Each decoder says how wide a column it reads, and the assignment builds columns to +exactly that width. + +A candidate is scored by the frame's cost with it sounding beside the picks its source already holds in +that frame. The channel's silence is always among the candidates. The picks add up to a mix: their +phase-averaged power spectra add, and so do the waveforms they are expected to render and their variances. + +Scoring runs in two stages. First, every candidate of one channel's kind is added to the mix and ranked by +the phase-independent spectral term. Then the best `top_k`, together with the kind's silence, are +re-scored with the full criterion. + +A candidate whose frames repeat one shape adds its waveform. With `find_best_phase` on, the waveform is +aligned to what the mix leaves of the target. Otherwise the candidate's library sample is added from its +start. Either phase stands in for the one the generator reaches when the frame is rendered. A candidate +whose frames show different stretches of a sequence renders whatever stretch its channel has reached. It +therefore adds its mean level and its variance, whatever `find_best_phase` says. + +The scored candidates form the channel's column, best first, with silence ahead of an equal cost. `top_k` +sets how many of them a wide decoder reads. ### 5.1 Assigning channels -A frame is assigned one pick at a time, for as long as a pick lowers a frame's cost: +A frame is assigned one pick at a time, for as long as a source may still take a channel and a pick lowers +a frame's cost. Each round takes the (source, channel) pair whose best candidate lowers its source's cost +the most, weighted by the energy of the source's frame, and adds that candidate to the source's mix. -``` -free = {channels the setup covers} -mix[source] = nothing, cost[source] = the cost of silence, for every source that sounds -while a source may still take a channel and free is non-empty: - for every source and every channel kind it may still take: - column = that kind's candidates scored with mix[source] sounding - take the (source, channel) whose column head sounds and lowers cost[source] the most, - weighted by the energy of the source's frame; stop when none does - add the head to mix[source], set cost[source] to the head's cost -give every channel still free to the first sounding source that may hold it, headed by silence -score every held channel once more with its source's other channels sounding -``` +When the picks end, every channel still free goes to the first sounding source that may hold it, headed by +silence. Then every held channel is scored once more with its source's other channels sounding. That last +pass lets a channel taken early fall silent where the later channels cover its sound, or sound where they +leave room. -A frame one channel renders whole therefore sounds one channel: once the triangle covers -a sine, adding a pulse or the noise raises the cost, and those channels hold their -silence. Where several channels share one generator kind at one drive, the lowest free channel -of that group represents it during scoring, so successive picks over one kind land on the -lowest free channel. A channel no pick took keeps its column, headed by its silence, so the -decoder may still sound it where the frames around ask for it; it counts against its -source's count, so no decoded frame sounds more channels than that count. The last -pass lets a channel taken early fall silent where the later channels cover its sound, or -sound where they leave room. A channel no source may hold **rests**: it holds its -channel's null instruction for that frame, which is what keeps every channel's stream in -step with the frames it describes. - -A source takes a channel in the frames its own audio reaches a level a channel can -render, and stands aside in the rest, so a frame it is silent in leaves its channels -resting. - -A classic single-file conversion is one source covering every enabled channel, so the -one mix answers the frame itself and every channel in each frame the source sounds in is -held, sounding or silent. Several sources, a precedence hierarchy, a drive per source -channel and a per-source count of channels sounding at once are the general case, -described in [Stems reconstruction](stems.md); there each mix answers one source's own -sound, which is what makes the channel a source wins carry that source's material. +A frame one channel renders whole therefore sounds one channel. Once the triangle covers a sine, adding a +pulse or the noise raises the cost, and those channels hold their silence. + +Where several channels share one generator kind at one drive, the lowest free channel of that group +represents it during scoring. Successive picks over one kind therefore land on the lowest free channel. + +A channel no pick took keeps its column, headed by its silence, so the decoder may still sound it where +the frames around ask for it. It counts against its source's count, so no decoded frame sounds more +channels than that count. + +A channel no source may hold is [**resting**](../glossary.md#resting). It plays its channel's null +instruction for that frame, which keeps every channel's stream in step with the frames it describes. A +source takes a channel in the frames its own audio reaches a level a channel can render, and stands aside +in the rest. A frame the source is silent in leaves its channels resting. + +A classic single-file conversion is one source covering every enabled channel. The one mix answers the +frame itself, and every channel in each frame the source sounds in is held, sounding or silent. Several +sources, a precedence hierarchy, a [drive](../glossary.md#drive) per source channel and a per-source +count of channels sounding at once are the general case, described in [Stems reconstruction](stems.md). +There each mix answers one source's own sound, so the channel a source wins carries that source's +material. ### 5.2 Greedy decoding -The greedy decoder plays each frame's best candidate, reading one candidate per -column. Each frame is then decided by its own cost alone, which is fast and -straightforward, and the instruction streams follow each frame's match wherever it -leads — audible as jitter even where every individual frame is well matched. +The greedy decoder plays each frame's best candidate, reading one candidate per column. Each frame is +decided by its own cost alone, which is fast and simple. The instruction streams follow each frame's match +wherever it leads. That is audible as jitter even where every individual frame is well matched. ### 5.3 Viterbi decoding -The Viterbi decoder weighs a frame's candidates against the frames around them. It -reads `top_k` candidates per column, the channel's silence kept among them, forming a -lattice of states over time, and -finds, per channel, the lowest-cost **path** through that lattice, where the path -cost combines: +The Viterbi decoder weighs a frame's candidates against the frames around them. It reads `top_k` +candidates per column, with the channel's silence among them, and forms a lattice of states over time. Per +channel, it finds the lowest-cost **path** through the lattice. The path cost combines: - the per-frame **match cost** (the criterion, as an emission cost), and -- a **transition cost** between consecutive frames that grows with what changes - between two instructions — turning a channel on or off, and changing pitch, volume - or timbre. +- a **transition cost** between consecutive frames that grows with what changes between two instructions: + turning a channel on or off, and changing pitch, volume or timbre. -Minimizing emission plus transition costs (the classic Viterbi dynamic program) -yields instruction streams that track the audio while changing only when the -improvement in match quality outweighs the cost of the change. The result is smoother -and more musical than the greedy output. It is the default. +Minimizing emission plus transition costs (the classic Viterbi dynamic program) yields instruction +streams that track the audio while changing only when the improvement in match quality outweighs the cost +of the change. The result is smoother and more musical than the greedy output. It is the default. -A resting frame reaches the decoder as a column of one, so a channel that no source -took sits in the path as the off state it is, and coming back on costs what any other -on/off change costs. A frame the decoder settles on a silent instruction is released to -the resting stem, so the resting stem id and the silence name the same frames, and a -channel decoded silent throughout stands by. +A resting frame reaches the decoder as a column of one, so a channel that no source took sits in the path +as the off state it is. Coming back on costs what any other on/off change costs. A frame the decoder +settles on a silent instruction is released to the resting stem, so the resting stem id and the silence +name the same frames. A channel decoded silent throughout is [standing by](../glossary.md#standing-by). ## 6. Refining the pitch -The catalog is built on the equal-tempered grid, so the matching can place a frame no closer than -the nearest semitone. The hardware is finer than that: a note reaches a channel as an 11-bit -divider, and one step of that divider spans **0.85 cents at A-0, 4 cents at C-3, 16 cents at C-5**, -reaching a whole semitone only around C-7, where the divider grid and the note grid meet. Everything -below that is room the matching leaves unused, and material that was never in A=440 equal -temperament — most recordings of most instruments — sits somewhere inside it. +The catalog is built on the equal-tempered grid, so the matching places a frame no closer than the nearest +semitone. The hardware is finer: a note reaches a channel as an 11-bit [divider](../glossary.md#divider). +One step of that divider spans **0.85 cents at A-0, 4 cents at C-3 and 16 cents at C-5**. It reaches a +whole semitone only around C-7, where the divider grid and the note grid meet. Everything below that is +room the matching leaves unused. Material that was never in A=440 equal temperament, which is most +recordings of most instruments, sits somewhere inside it. -`sampletones_core.reconstructions.reconstructor.refinement` spends that room, after the decoder has -settled which note each frame plays and before the frames are rendered. It spends it where the run -asks: a stem entry names the channels it carries toward its own recording, so one recording's bass -line can land on its exact tuning while another's lead keeps the grid. +The **refinement** uses that room. It runs after the decoder has settled which note each frame plays and +before the frames are rendered. It works where the run asks: a stem entry names the channels it carries +toward its own recording, so one recording's bass line can land on its exact tuning while another's lead +keeps the grid. ### 6.1 Reading rather than searching -The refinement does not search. Two measurements settle why: - -- Against a **matched** candidate the criterion answers a detune smoothly and monotonically — a - 25-cent error costs about 0.09 where a 50-cent error costs about 0.40. Against a **realistic** - target, where the candidate cannot match the timbre, that response is a small ripple on a - timbre-dominated floor with many local minima, and taking the lowest-cost divider over a sweep - lands 15–30 cents from the truth. -- Searching also costs what the library exists to avoid. Scoring one extra candidate per frame - means rendering it and extracting its feature, which measures around **2.1 s per second of - audio** — more than a whole conversion of the same audio. - -So the answer is read out of the transform instead. `sampletones_core.fft.instantaneous` takes the -**phase** the constant-Q transform already computes and `calculate_cqt_spectrum_columns` discards. -A partial standing between two bin centers still advances its phase at its own rate, so comparing -that advance across two columns against the rate the bin itself turns at states the partial's -frequency far more finely than the bins are spaced. Reading the first few harmonics of the note the -decoder chose, each weighted by the energy behind it and each settled against the fundamental the -harmonics below it agreed on, places the note **within a tenth of a cent** across the whole range. - -The reading also states how much of the frame stands behind it — the share of the column's energy -its harmonics hold. A pitched frame reads around 0.5, a frame sharing the channel with another tone -around 0.3, and noise around 0.04, so one threshold separates the frames worth bending from the -frames with no pitch to read. +The refinement reads the pitch out of the recording. Searching for it has two problems. + +- The criterion is a poor guide to tuning on real material. Against a **matched** candidate it answers a + detune smoothly and monotonically: a 50-cent error costs about four times what a 25-cent error does. + Against a **realistic** target, where the candidate cannot match the timbre, the response is a small + ripple on a timbre-dominated floor with many local minima. Taking the lowest-cost divider over a sweep + then lands 15–30 cents from the truth. +- Searching costs what the library exists to avoid. Scoring one extra candidate per frame means rendering + it and extracting its feature. On the same audio and the same machine, that alone took longer than the + whole conversion. + +The reading comes from the transform instead, from the **phase** the constant-Q transform already computes +and the spectrum discards. A partial standing between two bin centers still advances its phase at its own +rate. Comparing that advance across two columns against the rate the bin itself turns at gives the +partial's frequency far more finely than the bins are spaced. The reading takes the first few harmonics of +the note the decoder chose. It weights each by the energy behind it and settles each against the +fundamental the harmonics below it agreed on. This places the note **within a tenth of a cent** across the +whole range. + +The reading also says how much of the frame stands behind it: the share of the column's energy its +harmonics hold. A pitched frame reads around 0.5, a frame sharing the channel with another tone around +0.3, and noise around 0.04. One threshold therefore separates the frames worth bending from the frames +with no pitch to read. ### 6.2 Landing the note, and holding it -A reading becomes a bend through the generator, which owns the divider geometry: `bend_toward` -answers with the divider steps that land the note nearest the frequency read, bounded by -`bend_range` — **half the gap to each neighboring note**. That bound is what leaves the refined -pitches gapless: note *n* covers `[(tₙ + tₙ₊₁) / 2, (tₙ + tₙ₋₁) / 2]`, and those windows tile the -divider range exactly, so every divider the notes span is reachable and none is claimed twice. +A reading becomes a bend through the generator, which owns the divider geometry. The generator answers +with the divider steps that land the note nearest the frequency read, bounded to **half the gap to each +neighboring note**. That bound leaves the refined pitches gapless. Note *n* covers +`[(tₙ + tₙ₊₁) / 2, (tₙ + tₙ₋₁) / 2]`, and those windows tile the divider range exactly, so every divider the +notes span is reachable and none is claimed twice. -A bend that followed every reading exactly would jitter, and jitter is more audible than the tuning -it chases. So the per-frame proposals are settled by a change-penalized walk, the same shape the -Viterbi decoder settles a note contour with: the cost of a bend is how far it stands from that -frame's reading, plus a toll on changing at all. The states a frame may take are the bends its -neighborhood proposed together with no bend, which keeps the walk to a handful of states even where -a note owns tens of dividers. +A bend that followed every reading exactly would jitter, and jitter is more audible than the tuning it +chases. The per-frame proposals are therefore settled by a change-penalized walk, the same shape the +Viterbi decoder uses to settle a note contour. The cost of a bend is how far it is from that frame's +reading, plus a toll on changing at all. The states a frame may take are the bends its neighborhood +proposed, together with no bend. That keeps the walk to a handful of states even where a note owns tens of +dividers. ### 6.3 What it costs, and what it leaves alone -The refinement enumerates no candidate, rescores nothing, and leaves the library, the per-frame -matching and the decoder's lattice exactly as they were. What it adds is one transform per -recording and a small walk per channel. - -What that transform costs depends on the machine, and the spread is wide: on a CUDA build it -disappears into the noise, while on a CPU build it is a measurable share of a short conversion — -a tenth or more, since the reading needs a handful of bins per frame and the transform computes -every bin the spectrum covers. Restricting it to the bins the chosen notes actually name is the -work `docs/development/bugs-and-todos.md` records under **Features**. - -A frame makes no proposal where it rests, where the stem holding it leaves that channel out, where -its channel is not pitched — the noise channel's sixteen periods have no finer grid — or where its -reading falls below the confidence threshold. A conversion that bent no note records both bend -dimensions as ones the channel governs, so it writes the instrument an unrefined run writes. - -Which recordings are carried, and on which channels, each stem entry states for itself in -`bends` — a subset of the channels it occupies, and of the three that load a divider. A channel a -stem leaves out keeps the note the matching chose, and a stem carrying nothing at all is never -read, so the transform is spent only where a bend comes of it. The settings below shape a bend -once it is asked for, and hold for a whole run. - -| parameter | default | notes | -|---|---|---| -| `generation.refinement.confidence` | 0.15 | the share of a frame's energy its harmonics must hold | -| `generation.refinement.change_weight` | 2.0 | divider steps of reading error worth avoiding one change | -| `generation.refinement.window` | 4 | the frames on either side whose readings a frame may settle on | +The refinement enumerates no candidate and rescores nothing. It leaves the library, the per-frame matching +and the decoder's lattice exactly as they were. It adds one transform per recording and a small walk per +channel. + +The transform's cost depends on the machine. On a CUDA build it is too small to measure. On a CPU build it +is a tenth or more of a short conversion, because the reading needs a handful of bins per frame and the +transform computes every bin the spectrum covers. Restricting it to the bins the chosen notes name is +recorded in [bugs and to-dos](../development/bugs-and-todos.md) under **Features**. + +The refinement keeps a bend on the reading alone. Scoring each bent candidate would repeat the +render-and-score cost described in [section 6.1](#61-reading-rather-than-searching), and the criterion +would only agree with the reading. + +A frame makes no proposal where it rests, where the stem holding it leaves that channel out, where its +channel is not pitched (the noise channel's sixteen periods have no finer grid), or where its reading +falls below the confidence threshold. A conversion that bent no note records both bend dimensions as ones +the channel governs, so it writes the instrument an unrefined run writes. + +Each stem entry says which recordings are carried, and on which channels, in `bends`: a subset of the +channels it occupies, and of the three that load a divider. A channel a stem leaves out keeps the note the +matching chose. A stem carrying nothing at all is never read, so the transform is spent only where a bend +comes of it. + +The settings under `generation.refinement` shape a bend once it is asked for: `confidence`, +`change_weight` and `window`. They hold for a whole run. [The configuration file](../formats/configuration.md) +lists them. ## 7. Rendering and reassembly -A reconstruction is its instruction streams. Each one is rendered back through its generator -(`sampletones_core.generators`), which carries oscillator phase across frames so -there are no clicks at frame boundaries, and resets it on a new note where -`reset_phase` says so; an "off" instruction yields silence for that channel and frame. Each -frame is rendered at the drive the source holding it gives its channel (see -[Stems reconstruction](stems.md)), and the per-channel renderings are summed into the final -approximation. - -The rendering is read on demand rather than carried beside the streams, so what a -reconstruction shows is what an export plays by construction, and a `.stn` states the -instructions, the per-frame ownership and the setup they were chosen under — a few megabytes -where the audio would be a few hundred. `Reconstruction.approximations` is that reading; the -coefficient from §3.4 is stored so the reconstruction and the original can be shown and played -on a common scale. +A reconstruction is its instruction streams. Each stream is rendered back through the channel that plays +it, as it stands. The channel carries the oscillator's phase across frames, so frame boundaries make no +clicks, and resets it on a new note where the settings say so. An "off" instruction gives silence for that +channel and frame. The per-channel renderings are summed into the final approximation. + +The audio is rendered on demand, so what a reconstruction shows is what an export plays. A `.stn` +therefore holds the instructions, the per-frame ownership and the setup they were chosen under, which is +far smaller than the audio would be. The working level from +[section 3.4](#34-the-working-level-coefficient) is stored too, so the reconstruction and the original can +be shown and played on a common scale. ## 8. Limitations -- **Dynamic range.** A single NES tonal channel spans roughly 25 dB from its - quietest to its loudest note, and the coefficient is one global scalar. Material - whose *useful* content spans a wider range than that (a long crescendo, a very - quiet passage under a loud one) cannot be fully captured: content far below the - working level falls under the quietest playable note and is rendered as silence. -- **CQT time resolution.** Because constant-Q analysis needs long windows at low - frequencies, low-pitched transients are inherently smeared in time under `cqt`; - `fft`/`logfft` localize time better at the cost of low-frequency resolution. -- **Per-channel independence in Viterbi.** Channels are decoded independently once the - assignment has settled their columns, which is fast but not jointly optimal across - channels. -- **Refinement needs a fundamental to read.** A frame carrying several pitches at once, or one - whose sound is unpitched, states no fundamental for its channel and keeps the note the matching - chose. The room a bend has also closes with pitch: a divider step is a whole semitone from around - C-7 up, so notes there sound where the grid puts them. - -## Appendix — key parameters and where things live - -Default configuration (44.1 kHz, 60 Hz change rate, channels pulse 1 + triangle + -noise): - -| parameter | default | notes | -|--------------------------|---------|----------------------------------------------------| -| frame length | 735 | `sample_rate / nes_frequency`, ~17 ms | -| spectrum method | `cqt` | `fft` / `logfft` / `cqt` | -| `transformation_gamma` | 0 | 0 = power spectrum, 100 = log | -| spectral / temporal weight | 0.8 / 0.2 | criterion blend | -| spectral distance | β-divergence | also `squared`, `absolute` | -| selector | Viterbi | `greedy` / `viterbi` | -| pitch refinement | per stem | bends each note onto the divider the source sounds | -| normalize / quantize | on / off | input preprocessing | - -Package map: - -| concern | package | -|---------------------------------|------------------------------------------------------| -| NES channel models | `sampletones_core.generators` | -| instruction value types | `sampletones_core.instructions` | -| windowing, spectra, features | `sampletones_core.fft` | -| candidate catalog | `sampletones_core.library` | -| scoring | `sampletones_core.reconstructions.criterion` | -| selection + assembly | `sampletones_core.reconstructions.reconstructor` | -| audio I/O and level | `sampletones_core.audio` | -| tracker export | `sampletones_core.exporters` | -| pitch refinement | `sampletones_core.reconstructions.reconstructor.refinement` | -| criterion calibration | `sampletones_tools.calibration` | -| analytic waveform synthesis | `sampletones_tools.synthesis` | +- **Dynamic range.** A single NES tonal channel spans roughly 25 dB from its quietest to its loudest note, + and the coefficient is one global scalar. Material whose *useful* content spans a wider range than that + cannot be fully captured. A long crescendo and a very quiet passage under a loud one are examples. + Content far below the working level falls under the quietest playable note and is rendered as silence. +- **CQT time resolution.** Constant-Q analysis needs long windows at low frequencies, so low-pitched + transients are smeared in time under `cqt`. `fft` and `logfft` localize time better at the cost of + low-frequency resolution. +- **Per-channel independence in Viterbi.** Channels are decoded independently once the assignment has + settled their columns. That is fast but not jointly optimal across channels. +- **Refinement needs a fundamental to read.** A frame carrying several pitches at once, or one whose sound + is unpitched, has no fundamental for its channel and keeps the note the matching chose. The room a bend + has also closes with pitch: a divider step is a whole semitone from around C-7 up, so notes there sound + where the grid puts them. + +## Appendix — the settings behind all this + +Every choice described here is a setting you can change. The +[configuration file](../formats/configuration.md) lists them with the values each one accepts, and the +shipped values are in `sampletones_core/configs/generation.yaml`. diff --git a/docs/concepts/stems.md b/docs/concepts/stems.md index abd943fa9..64c34a563 100644 --- a/docs/concepts/stems.md +++ b/docs/concepts/stems.md @@ -1,473 +1,169 @@ # Stems reconstruction -This document explains how one reconstruction is assigned across several stems. -Consult it when changing the stems assignment algorithm, its configuration, the -per-stem record a reconstruction carries, or the way the application loads, -names, reveals, and plays the recorded stems. The single-sample pipeline this -builds on is described in [Reconstruction](reconstruction.md), and the stored -record in [Reconstructions](../formats/reconstructions.md). - -A stems reconstruction converts several audio stems at once. Each stem is matched -against the instruction library on its own; within each frame, the channels are -handed to the stems one pick at a time, following a precedence hierarchy. The -result is one reconstruction whose `stems_data` records, per channel and frame, -which stem's stream plays. +This document explains how the four channels are shared out when a reconstruction is built from several +stems. Read it before changing how stems share channels. You can read it without the source code. +[Reconstruction](reconstruction.md) describes the single-sample pipeline it builds on. +[Reconstructions](../formats/reconstructions.md) documents the stored record. +[Stems in the application](../development/application/stems.md) covers what the application does with a +stems reconstruction: the Stems card, an edit and a removal. + +A stems reconstruction converts several audio [stems](../glossary.md#stem) at once. Each stem is matched +against the [instruction library](../glossary.md#instruction-library) on its own. Within each +[frame](../glossary.md#frame), the channels are handed to the stems one [pick](../glossary.md#pick) at a +time, following a precedence hierarchy. The result is one reconstruction whose `stems_data` records, per +channel and frame, which stem's stream plays. + +The terms [level](../glossary.md#level-stems), [drive](../glossary.md#drive), +[mix](../glossary.md#mix), [column](../glossary.md#column) and [resting](../glossary.md#resting) are +defined in the glossary. ## Principles ### 1. Every conversion is a stems conversion -Below the conversion job there is one pipeline and one entry point. A job names -the recordings it mixes, the stems setup that hands their channels out, and the -file it writes; a conversion from a single file is the job whose setup holds one -stem over every enabled channel. What the reader chose stays above that line: -the application decides how many jobs a request makes and what setup each -carries, and a batch is many single-source jobs rather than a mode of its own. +Below the conversion job there is one pipeline and one entry point. A job names the recordings it mixes, +the stems setup that hands their channels out, and the file it writes. A conversion from a single file is +the job whose setup has one stem over every enabled channel. What the reader chose stays above that line: +the application decides how many jobs a request makes and what setup each carries. A batch is many +single-source jobs and is not a mode of its own. -This is what lets a per-source channel set, the drive each channel is pushed at, -a count of channels one source may sound at once and a hierarchy reach every -conversion alike, and what keeps the classic run from being a second path that -has to be kept in step. +One pipeline lets a per-source channel set, the drive each channel is pushed at, a count of channels one +source may sound at once and a hierarchy reach every conversion alike. It also keeps the classic run from +becoming a second path that has to be kept in step. ### 2. A stem is matched against its own recording -Every stem is loaded, padded to the longest stem's length, and framed on its own, -so a stem's picks are scored against the sound that stem contributes and the -channels it wins carry that recording. Ownership and content then say the same -thing: a stem heard on its own plays what was recorded on it. +Every stem is loaded, padded to the longest stem's length, and framed on its own. A stem's picks are +therefore scored against the sound that stem contributes, and the channels it wins carry that recording. +Ownership and content say the same thing: a stem heard on its own plays what was recorded on it. -The mix keeps the two jobs it answers: the whole set is scaled by one factor drawn -from the peak of its sum, which holds the stems at the balance they were captured -in, and the working-level coefficient is measured on that mix, exactly as for a -single file. The reconstruction the run assembles is the sum of the stems' -approximations, which approximates the mix because each part approximates its part. +The whole set of recordings is scaled by one factor drawn from the peak of its sum, which holds the stems +at the balance they were captured in. The working-level coefficient is measured on that sum, exactly as +for a single file. The reconstruction the run assembles is the sum of the stems' approximations. It +approximates the summed recordings because each part approximates its part. -### 3. A stem sounds at the drive its settings give each channel +### 3. A drive reaches for a louder instruction -A stem's settings name a drive per channel it holds: the factor the channel's -output is scaled by, `1.00` standing at the level the library is calibrated to. -The drive reaches the matching as well as the rendering — candidates are scored -at it and the winning instruction is recorded at it — so a channel pushed harder -is answered by the instructions that carry the recording at that level, and the -frame that reaches the mix sounds at the level it was chosen for. A drive answers -for one channel of one stem, so raising it lifts that part of the mix while the -other stems and the stem's other channels stand where they are. +A stem's settings name a drive per channel it holds. `1.00` is the level the library is calibrated to. The +drive belongs to the conversion alone. Every candidate is read at unit drive, so the instruction winning a +frame is the one that sounds it at the drive. A channel pushed harder is answered by louder instructions, +up to the loudest that channel holds. Past that the channel saturates, which is the effect a drive is used +for. + +A drive applies to one channel of one stem. Raising it lifts that part of the mix, and the other stems and +the stem's other channels stay where they are. Afterwards a frame sounds the instruction it carries, so +nothing reads the drive once the conversion is done. The instruction is the record of the level the run +reached. ### 4. A stem sounds where its recording sounds -A stem takes a channel in the frames its own recording reaches a level a channel -can render, and stands aside in the rest. A channel a passing stem leaves free goes -to a stem that does sound there, or rests. This is what keeps a recording quiet -through a passage from sounding that passage on the channels it holds elsewhere. +A stem takes a channel in the frames where its own recording reaches a level a channel can render, and +stands aside in the rest. A channel a passing stem leaves free goes to a stem that does sound there, or +rests. That keeps a recording that is quiet through a passage from sounding the passage on the channels it +holds elsewhere. ### 5. One pick at a time, while a pick helps -A pick scores each eligible stem's candidates by the cost of that stem's own frame -with the candidate sounding beside the stem's earlier picks, with the same two-stage -criterion the single-sample pipeline uses (`FrameMatcher`), takes the winning offer -across the active level, adds it to that stem's mix, and consumes the channel. Picks -continue while an offer lowers a frame's cost and counts and free channels remain. A -channel no pick took goes, silent, to the first sounding stem in hierarchy order that -may still hold it, and every held channel is then scored once more with its stem's other -channels sounding. Each stem carrying a mix of its own is what keeps its later picks -from re-approximating what its earlier picks already cover, while leaving what the -other stems sound out of it. +A pick scores each eligible stem's candidates by the cost of that stem's own frame with the candidate +sounding beside the stem's earlier picks. It uses the same two-stage criterion as the single-sample +pipeline. The best candidate across the active level wins, is added to that stem's mix, and takes the +channel. + +Picks continue while a candidate lowers a frame's cost, a stem's count leaves it room, and free channels +remain. A channel no pick took goes, silent, to the first sounding stem in hierarchy order that may still +hold it. Then every held channel is scored once more with its stem's other channels sounding. + +Each stem has a mix of its own. That keeps a stem's later picks from re-approximating what its earlier +picks already cover, and it leaves what the other stems sound out of the stem's mix. ### 6. A frame is answered whole -Every channel the setup covers leaves a frame either held by a stem or **resting**. A -resting channel holds its channel's null instruction over a silent frame and -records the resting stem id, so instruction streams, rendered approximations and -the per-frame stem record all run parallel to the frames they describe: frame -*i* of a channel is frame *i* of the recording. A frame the decoder settles on a -silent instruction records the resting stem id too, so the id and the silence name -the same frames. A channel that rests through every frame stands by instead, carrying -no stream at all. +Every channel the setup covers leaves a frame either held by a stem or **resting**. A resting channel +holds its channel's null instruction over a silent frame and records the resting stem id. Instruction +streams, rendered approximations and the per-frame stem record therefore all run parallel to the frames +they describe: frame *i* of a channel is frame *i* of the recording. A frame the decoder settles on a +silent instruction records the resting stem id too, so the id and the silence name the same frames. A +channel that rests through every frame is [standing by](../glossary.md#standing-by) and has no stream at +all. -This is what makes a channel count and a hierarchy usable. Without it, a frame a -count left unclaimed would shorten that channel's streams and carry its later -frames early, so what the channel plays would drift out of step with the -recording it was matched against. +This is what makes a channel count and a hierarchy usable. Without it, a frame a count left unclaimed +would shorten that channel's streams and carry its later frames early. What the channel plays would drift +out of step with the recording it was matched against. ### 7. Ownership and decoding compose -The assignment answers *which stem owns which channel this frame*; the decoder -answers *what that channel plays across frames*. Each held channel leaves the frame -with a column of candidates, its silence among them, as wide as the configured -decoder reads, and the decoder chooses one candidate per frame from those columns — -greedily, or along the lowest-cost path through the whole lattice. A resting frame reaches the -decoder as a column of one, so a channel a count left free sits in the path as the -off state it is. See [Reconstruction §5](reconstruction.md) for the decoders -themselves. +The assignment answers *which stem owns which channel this frame*. The decoder answers *what that channel +plays across frames*. Each held channel leaves the frame with a column of candidates, its silence among +them, as wide as the configured decoder reads. The decoder chooses one candidate per frame from those +columns, greedily or along the lowest-cost path through the whole lattice. A resting frame reaches the +decoder as a column of one, so a channel a count left free sits in the path as the off state it is. +[Choosing instructions](reconstruction.md#5-choosing-instructions) describes the decoders. -### 8. Precedence orders, mode alternates +### 8. Levels pick in order, and the mode sets how -The hierarchy groups stem ids into levels that pick in the listed order. In -`strict` mode a level exhausts its stems' channel counts before the next level -picks; in `round_robin` mode the levels take turns, granting every level's stems -one channel per round, and a run lasts as many rounds as the largest count among -the stems sounding. Both modes hold every stem to the count its own settings -name, and a stem holding fewer channels than that runs out of channels first. +The hierarchy groups stem ids into levels that pick in the listed order. In `strict` mode a level uses up +its stems' channel counts before the next level picks. In `round_robin` mode the levels take turns, +granting every level's stems one channel per round, and a run lasts as many rounds as the largest count +among the stems sounding. Both modes hold every stem to the count its own settings name. A stem holding +fewer channels than that count runs out of channels first. ### 9. A level's channel goes to the stem with the most to render -A cost is a fraction of its own recording's energy, so two stems' costs stand on -different scales and comparing them alone would hand a channel to whichever -recording is easiest to approximate. Within a level, an offer is therefore ranked -by how far its head lowers the stem's frame cost, weighted by the energy behind it, -so the channel reaches the stem whose sound it covers most. Precedence between levels -stays the hierarchy's, which is what a reader arranges the levels to say. +A cost is a fraction of its own recording's energy, so two stems' costs stand on different scales. +Comparing them alone would hand a channel to whichever recording is easiest to approximate. Within a +level, a candidate is therefore ranked by how far it lowers its stem's frame cost, weighted by the energy +behind it. The channel reaches the stem whose sound it covers most. Precedence between levels stays the +hierarchy's, because that is what a reader arranges the levels to say. ### 10. Ties resolve deterministically -Equal offers go to the stem earlier in level order. Channels of one kind at one -drive are scored as one column and resolve to the lowest free channel of that -group, so successive picks over one kind land on the lowest free channel and a -rerun assigns the same way every time. Two channels of a kind a stem drives -differently answer for themselves, since each renders the recording at its own -level. +Equal candidates go to the stem earlier in level order. Channels of one kind at one drive are scored as +one column and resolve to the lowest free channel of that group. Successive picks over one kind therefore +land on the lowest free channel, and a rerun assigns the same way every time. Two channels of a kind that +a stem drives differently are scored separately, because each renders the recording at its own level. ### 11. The single-sample case stays exact -One stem covering every enabled channel at unit drive, sounding as many channels -at once as it holds, is the single-sample reconstruction. Property tests hold the -assignment against an independent restatement of the frame objective that scores -every candidate alone — -identical choices, instructions, costs and contributions — so the one pipeline serves -the single-sample case exactly as it stands. +One stem covering every enabled channel at unit drive, sounding as many channels at once as it has, is the +single-sample reconstruction. Property tests hold the assignment to an independent restatement of the +frame objective that scores every candidate alone. The one pipeline therefore serves the single-sample +case exactly. ### 12. The working level follows the covered channels -The mix is scaled so its typical frame plays at the full-scale RMS level of the -quietest tone channel the setup covers, or of the quietest covered channel when it -covers no tone channel (see [Reconstruction §3.4](reconstruction.md)). One channel -renders that level whole, so a run sounding fewer channels targets a level its -channels reach, and -every setup measures against the channel a single recording would be answered by. - -## Mechanics - -A request becomes jobs through `reconstructions.converter`: a `ConversionPlan` -answers with the `ConversionJob`s it divides into, resolved against the -configuration the run uses. `GroupConversion` mixes the recordings it is given -into one job; `BatchConversion` gives each gathered recording a job of its own, -carrying the setup that recording's own row holds and the folder whose tree its -reconstruction mirrors; and `DirectoryConversion` scans a folder into one -single-source job per audio file, which is what the command line converts a -directory as. `ReconstructionConverter` runs those jobs across its worker pool -and reports the reconstructions written. - -`StemsConfig` (`reconstructor/stems/configs/`) is the setup: the entries and the -precedence hierarchy with its mode. An entry is an id and the `StemSettings` its -recording is converted with — the channels it may occupy, which of those it -carries toward the divider it really sounds, the drive on each channel it holds, -and how many of them it may sound at once. Every per-recording choice being a -field on those settings is what lets the list a reader sets a run up in, the -entry the run records, and a later reader of that record all state the same -thing. `StemSettings` validates itself — a drive for exactly the channels held, -each within its bounds, and a count of at least one — and `StemsConfig` holds the -ids unique and the hierarchy naming every entry exactly once, so an inconsistent -setup can be neither built nor stored; it derives the views the run reads -(`entries_by_id`, `covered_channels`). `StemSettings.covering(channels)` is the -usual settings over a channel set: those channels, the tone channels among them -bending, unit drive on each, and all of them sounding at once. - -The assignment lives in `reconstructor/stems/assignment/`: - -- `assign_frame` takes this frame of every stem, keyed by stem id, validates the - setup against the run's channels, and answers the frame whole: the picks in the - order they were made, each with its candidate column, together with the channels - left resting; -- `AssignmentSession` carries one frame's progress — each stem's `FrameMix` and frame - cost, the free channels, the per-stem counts, and the stems sounding in the frame — - and runs the hierarchy's mode, then settles the declined channels and scores every - choice once more. It ranks a level's offers by `StemOffer.improvement`, which the - matcher measures through `score_column`, `mix_cost` and `reference_energy`; -- `TrackAssignment` gathers the frames into what the rest of the run reads: the - lattice each channel offers the decoder, and the stem owning each of its - frames. - -`column_groups` (`assignment/columns.py`) gathers the channels a stem still has -free into one group per generator class and drive, the lowest free channel of a -group standing for it. A stem driving its two pulses alike therefore scores them -as one column, as a run without drives does, and a stem driving them apart scores -each. `AssignmentSession` reads a stem's count and its drive per channel from -that stem's own entry, and caches a scored column under the stem, the generator -class and the drive it was scored at. - -The drive enters the run where the level matters. `CandidateProvider` serves the -library's powers, waveforms and moments as they stand, and `FrameMatcher` scales -them by the drive a column is scored at — powers and variances by its square, -waveforms and means by the drive itself — so a run at unit drive costs what a run -without drives costs. `render_streams` -(`reconstruction/rendering.py`) then renders each frame at the drive the stem -owning that channel gives it, reading the drive off the recorded entry, and a -frame no recording holds renders at unit drive. Scoring and rendering therefore -stand at one level: the instruction a frame records is the one chosen for the -sound that frame makes, whenever that sound is read. - -`Reconstructor.reconstruct` loads the sources through `load_stems`, which brings -them to one length and one scale, measures the working level on their mix, frames -each of them, assigns every frame, releases the channels that rested throughout, -decodes the remaining lattices, and reads the decoded streams into the state in -frame order. - -`STEM_ACTIVITY_FLOOR` is the level a stem's frame reaches to take a channel: the -quietest note any channel renders, measured against the working level. - -The record stored in a reconstruction (`stems_data`) holds the stems setup the -assignment was made under, one `StemSource` per entry — the recording's name and -the file it was read from — and, per channel, the stem id holding each frame, -parallel to the instruction streams. Every reconstruction carries one. Together -with the instruction streams it is the whole of what a `.stn` states: -`Reconstruction.approximations` reads the sound from the pair, and -`Reconstruction.audio_filepath` reads the locations off the sources, answering -with none once any recording has lost its own. - -The stems setup is built per conversion from the sources and the reader's -choices and travels with the job; it is part of the request rather than of the -standard configuration. A source the reader left holding no channel takes no part: -the recordings and the entries are derived in one pass, so such a source reaches -neither, and the target stays what the covered channels can render. The assignment -is greedy per frame: continuity of *who* owns a channel across frames, and playback -that decides per frame on the recorded streams, are future work. - -## The recorded stems in the application - -A stems reconstruction records one source per entry, each naming its recording and -the file it was read from. The application reads the locations through -`source_paths`: one path for a single source, the tuple for stems, and empty once -the reconstruction is detached from its origin. The names stay whatever happens to -the locations, so a detached document still says which recordings it was built -from. - -Opening the document loads the recorded stems through `load_stems`, the same call -the conversion loads them with, so each one carries the level it holds in the mix -and a stem heard on its own sounds at that level. The mix of them is the original -audio the source toggle and the waveform offer, computed fresh on every load. A -recorded stem absent or unreadable on this machine follows the single-source rule: -the whole original is unavailable, the approximation stands on its own, and the -application names the first missing path in its dialog. - -The document's name follows the naming rules in -`sampletones_core.reconstructions.naming`, applied to the recorded paths in -order: a single source names the document after the file's stem, several stems -sharing one directory name it after that directory, and paths sharing no -directory fall back to the `.stn` filename. - -The reconstruction tab names every recorded path on the Stems card, one row per -stem, each row carrying its own full-path tooltip and revealing its recording on -a click. The Audio source panel keeps the reconstruction's own file and the -choice between the two waveforms. Locating reveals every recorded path at once: -a Linux file manager offering `org.freedesktop.FileManager1` opens one window -with every stem selected, and every other environment opens one window per -directory holding them. - -## The stems card - -The reconstruction tab's Stems card turns the recorded assignment into a -listener the user can steer. It draws the same list the converter's card draws: -each row carries one stem under the level it was picked on, named by its -recording, with a leading master box and a colored box on every channel the -stem holds frames on. A setup line above the rows names the assignment's -hierarchy mode, and a **Collapse levels** toggle draws every row -in one table where the banding is in the way. Ticking a box admits that stem's -frames on that channel to everything the tab plays and exports; unticking -silences them. - -### Principles - -1. **Selection filters what plays, shows what it filters, and scopes what is - edited.** A ticked set projects the document rather than mutating it: the - waveform shows the ticked frames alone, the reconstruction toggle plays them - mixed, original playback plays the recordings heard anywhere mixed, the - instruments panel draws the envelopes of that same part beside the figures - measuring it, and both WAV export and instrument export write what stands on - screen. Each answer derives from the recorded per-channel assignment, so a - stem heard on one channel keeps its samples there and stays quiet on the - next. The same set says which frames an instrument edit writes — see - [Editing a stems reconstruction](#editing-a-stems-reconstruction). - - Two rules shape the reading. **A filtered reading states the frames it leaves - out rather than dropping them**, so the envelopes, the waveform and the record - line up column for column, and **it ends where it last sounds**, so a channel - every recording is left out on reads as standing by — its plot empty, its - figures at nothing and its instrument written nowhere. Together they make what - a reader sees, hears, edits and exports one and the same part of the document. -2. **A box stands where the choice reaches something.** A stem draws a box on a - channel exactly where the record gives it a frame there, so every box the card - offers changes what is heard. A stem the picker never chose offers none, and - its row reads as holding no frames. The frames a reader wrote by hand answer - to no recording, so they gather in a row of their own that reads and behaves - like any other. -3. **Every stem starts heard everywhere it holds frames.** A freshly opened - stems reconstruction ticks every box, which answers the full waveform and the - full original — the unfiltered document. -4. **The global channel choice takes precedence.** A channel switched off for - the whole reconstruction mutes its column while leaving every value where the - reader put it, so switching the channel back on restores the per-stem choice - intact. The two compose by construction: the global choice filters the - partials, the stems choice the approximations. -5. **The selection follows the open document.** The card lives with the - reconstruction it describes: opening a document seeds the rows and the ticked - boxes, a regenerated reconstruction keeps what the reader chose and ticks the - channels a stem newly reaches, and closing the document empties the card. -6. **Listening choices stay out of the document.** The ticked set is session - state, like every choice that shapes what is heard — see - [Playback](../development/application/playback.md). Saving the reconstruction records - the assignment, never the selection. So is the banding: collapsing the levels - changes how the card draws, never what it describes. -7. **Removing a recording edits the document.** Where a box steers listening, - the remove button rewrites what is described: the change is asked about first - and recorded in the project history, and what it releases is stated under - [Editing a stems reconstruction](#editing-a-stems-reconstruction). A - reconstruction holds at least one recording, so the last row standing keeps - its button held back. -8. **The ribbon names what the record holds.** Under the waveform runs a lane per - channel in play, divided into the stretches one recording holds throughout and - painted in that recording's color, with a resting stretch showing the ground. - Each lane stands in a row of its own, marked with its channel's letter in that - channel's color, so a bar's color answers for the recording while the letter - beside it answers for the channel. The recordings take a range of their own, so - a color never reads as a channel's. A recording takes its color from the place - it holds on the record, so one recording reads alike wherever it is drawn. The - lanes stand only where more than one recording is in play, since a document - answering to a single recording has nothing to tell apart, and a stretch - outside what the reader hears reads as a rest — the ribbon states what is - heard, like every other reading. Each row of the card leads with a square in - the color its recording is painted in, so a stretch on screen answers to a - name at a glance. - -### Mechanics - -`ReconstructionData.partials_for` and `ReconstructionData.waveform_data` take a -`StemSelection` — the stems each channel keeps — and zero the unselected frames -per channel before mixing (`filter_approximations` in -`sampletones_core.reconstructions.reconstruction.stems`), keeping every array at -its unfiltered length, so a filtered mix aligns with the unfiltered one sample -for sample. `original_mix_for` mixes the recordings of the stems heard on any -channel. `ReconstructionPanelLogic` holds the channels each stem is heard on and -re-answers the stems view model, the waveform, and the audio data whenever the -choice changes; the coordinator wires the card's `on_stem_channels_changed` hook -to that handler. A reconstruction that records one source presents a single row -for its recording, and one that records no source shows the card's empty -state. - -Removal runs through `without_stem` -(`sampletones_core.reconstructions.reconstruction.stems.removal`), which returns -a fresh reconstruction holding what the rule under -[Editing a stems reconstruction](#editing-a-stems-reconstruction) leaves. The tab -coordinator hands the result on as a `ReconstructionEdit`, the payload both a -regenerated instrument and a removed recording travel as, so one path rebinds the -open document and records the edit against the project history. - -## Editing a stems reconstruction - -A conversion answers, per channel and frame, which recording plays there. Everything a reader -does to the document afterward stands on that answer: the instruments panel rewrites what a -channel plays, the stems card chooses what is heard, and the remove button takes a recording -out. This section states the account of ownership all three keep, and the rules each gesture -follows. Consult it when changing an edit path, the per-frame record, or what the card offers. - -### Principles - -1. **A frame has exactly one owner.** Every frame of a channel in play is held by one - recording, by the reader's own hand, or by nobody. The hardware reads one instruction per - channel per frame, so this is what the channel allows rather than a convention the code - adopts, and it is what makes the per-frame record a partition of the channel's frames. The - resting stem id names the frames nobody holds; the **authored** stem id names the frames the - reader wrote. - -2. **Rest and silence name the same frames.** A frame rests exactly when its instruction is - silent, which the conversion establishes and every later gesture keeps. A reader looking at - a silent frame and a reader looking at the record therefore learn the same thing, and the - rules below follow from it rather than choosing around it. - -3. **An edit rewrites what a frame plays, and leaves who plays it.** Ownership answers a - question an edit asks nothing about, so a frame carries its owner through any change to its - instruction. A frame takes an owner by coming into play and releases it by falling silent. - This is what lets a reader shape a recording's part while the document keeps its account of - where that part came from. - -4. **A reconstruction records what is played and reads what is heard.** The document holds the - instruction each channel plays per frame, the recording behind each of those frames, the - setup they were chosen under and the working level. Its sound is read from those through the - generators, at the drive each frame's owner gives its channel, which is one answer serving - the waveform, playback, an export and the mixed approximation alike. A document therefore - states what it describes, and every gesture below is complete once it has settled the frames. - -5. **What is heard is what is edited.** The channels a recording is ticked on are one choice - serving two readings: the frames the waveform draws, and the frames an edit writes. A - recording switched off on a channel reads there and stays as it stands, so a reader reaches - one recording's part at a time through the card already in front of them. - -6. **The setup is the conversion's, and a removal alone rewrites it.** The entries, the levels, - the channels each recording may occupy, its bends, its drives and its count record how the - document was made. An edit leaves all of them as they stand. Taking a recording out is the - one gesture that changes them. - -7. **Detaching drops where a recording lives and keeps who played what.** A recording's name and - the frames it holds belong to the document; its location on this machine belongs to this - machine. The record states both, so detaching lets each location go and keeps every name — - a reconstruction embedded in a project therefore keeps a working stems card. - -### What the record holds - -The per-frame record answers for every channel in play, and these hold after every gesture: - -- a channel in play carries one owner per frame of its stream, each naming a recorded entry, - the resting stem id or the authored stem id; -- a channel standing by carries no stream and no record at all; -- a frame rests exactly where its instruction is silent; -- a frame a recording holds lies on a channel that recording's settings occupy, while an - authored frame answers to no settings; -- an entry holding no frame anywhere stays on the record, and its row reads as holding none. - -### What an edit carries - -An edit hands one channel a fresh set of envelopes, which become that channel's stream. Each -frame's owner follows from the frame it was and the frame it becomes: - -| the frame | becomes | -|---|---| -| sounding before and after | its owner, unchanged | -| sounding, edited silent | resting | -| resting, edited into play | authored | -| resting, edited silent | resting | -| written past the end of the stream | authored where it sounds, resting where it stays silent | -| dropped from the end of the stream, within the scope | gone, together with its ownership | -| dropped from the end of the stream, outside it | standing, in what it plays and who plays it | - -Rest and silence naming the same frames is what makes the table total: a frame's owner before -the edit already says whether it sounded, so the three lines above the last three cover every -frame the edit keeps. - -An edit shortens a channel as far as its scope reaches. The stream therefore runs through the -last frame standing outside that scope, the frames the edit did reach rest along the way, and a -stream edited down to no frame leaves its channel standing by wherever the scope covered every -one of them. A channel written back into play comes back wholly authored. The setup, the -recorded sources, the identifier, the configuration and the working level stand throughout. - -**The scope an edit writes in** follows principle 5: a frame accepts a gesture where its owner -is ticked on that channel, and where it rests. The frames a reader wrote answer to a row of -their own, so they are ticked and reached like any recording's; a rest belongs to no row, which -is what lets a gesture write a note into silence. Every other frame draws dimmed and reads as it -stands. The scope is the same selection the waveform filter reads, so one state answers both, -and a reader narrowing what they hear narrows what they change with it. - -### What a removal releases - -Taking a recording out (`without_stem`) releases the frames it held: each states its channel's -silent instruction and takes the resting stem id, and a channel the removal empties stands by. -The entry and its source leave the record, a level the removal empties collapses, and the ids -of the recordings that stay are left alone, so the record and a reader's selection both stay -valid. A reconstruction holds at least one recording, so the last one standing keeps its place. - -A frame that stays keeps the instruction it played and the recording that held it. Its samples -follow from principle 4: a channel the removal reached is read afresh, so its oscillator runs -through the silence the removal left rather than through the notes it took away. - -An edit made before the removal changes nothing about it: the channel carries its record -whatever was written into it, so the frames the recording held are released the way they would -have been. The frames the reader authored answer to no recording and stand through every -removal. +The summed recordings are scaled so their typical frame plays at the full-scale RMS level of the quietest +tone channel the setup covers, or of the quietest covered channel when the setup covers no tone channel +(see [the working level](reconstruction.md#34-the-working-level-coefficient)). One channel renders that +level whole, so a run sounding fewer channels targets a level its channels reach. Every setup measures +against the channel a single recording would be answered by. + +## The setup and the record + +The **setup** is the entries and the precedence hierarchy with its mode. An entry is an id and the +settings its recording is converted with: + +- the channels it may occupy, +- which of those it carries toward the divider it really sounds, +- the drive on each channel it holds, +- how many of those channels it may sound at once. + +Every per-recording choice is a field on those settings. The list a reader sets a run up in, the entry the +run records and a later reader of that record therefore all say the same thing. The settings check +themselves: a drive for exactly the channels held, each within its bounds, and a count of at least one. +The setup keeps the ids unique and the hierarchy naming every entry exactly once, so an inconsistent setup +can be neither built nor stored. + +The setup is built per conversion from the sources and the reader's choices, and it travels with the job. +It is part of the request and not of the standard configuration. A source the reader left holding no +channel takes no part: it reaches neither the recordings nor the entries, and the target stays what the +covered channels can render. + +A stem's frame has to reach a floor before it can take a channel: the quietest note any channel renders, +measured against the working level. Below that, the frame has nothing audible to contribute. + +The record stored in a reconstruction has the setup the assignment was made under. It also has one source +per entry, naming the recording and the file it was read from, and, per channel, the stem that holds each +frame, parallel to the instruction streams. Every reconstruction has one. Together with the instruction +streams it is the whole of what a `.stn` says. See [Reconstructions](../formats/reconstructions.md). + +The assignment is greedy per frame, so which stem owns a channel can change from one frame to the next. diff --git a/docs/development/application/browser.md b/docs/development/application/browser.md index 306b58657..1452d05e9 100644 --- a/docs/development/application/browser.md +++ b/docs/development/application/browser.md @@ -1,282 +1,87 @@ # The Reconstruction Browser -This document governs the tree of reconstructions the **Reconstruction** and **Sequencer** tabs -share: how a reconstructions directory becomes rows, what a row stands for, and what it answers. -Consult it when changing what the browser lists, how a row reads, or what a click on one does. It -complements `docs/development/architecture.md` (layering and ownership) and -`docs/development/guidelines.md` (coding rules). +This document governs the tree of reconstructions the **Reconstruction** and **Sequencer** tabs share: how a reconstructions directory becomes rows, what a row represents, and what it answers. Consult it when changing what the browser lists, how a row reads, or what a click on one does. It complements [`architecture.md`](../architecture.md) (layering and ownership) and [`guidelines.md`](../guidelines.md) (coding rules). + +Four terms recur. A **row** is one line of the tree. A **heading** is a row the browser writes itself, such as a frequency pair or a source folder, as opposed to a row for a path on the disk. A **branch** is one of the two top-level views of the same reconstructions: the configuration branch lists them as the disk holds them, and the sample branch groups them by the audio they were made from. A **mode** is a state a browser can be in, such as showing favorites only. --- ## Principles -1. **One reading of the disk feeds every view.** A refresh walks the reconstructions directory once - into a `ReconstructionScan`, and every branch is built from that record. The views therefore agree - about what exists by construction, and a folder name is parsed into its configuration fields once - per refresh. -2. **The model carries the shape; the panel carries the widgets.** Which rows exist, what they are - called, which of them fold together and in what order they sit are decided on the tree. Both tabs - render one model, so they show one shape, and each rule is exercised without a window. -3. **A row's identity is its path; its name is a label.** Favorites, the context menus, copy-path, - playback and opening a reconstruction all key on `filepath`. That is what frees a name to be - rewritten — a configuration directory renamed to its channel abbreviation, a chain of headings - joined into one row, a colliding label marked with its configuration hash. -4. **The browser writes the headings the disk states rather than holds.** A frequency pair, a - transformation, a source folder, one source audio: each becomes a row that carries no path of its - own. What such a row offers follows from the subtree beneath it. -5. **One thing may stand in several places.** A reconstruction is listed by the configuration that - produced it and again by the audio it was made from, so an action on the thing rather than on the - row asks for every row standing for it (`Tree.find_nodes`, `BrowserManager.nodes_at`) and hands - them to both tabs. -6. **Per-row work happens off the main thread.** A rebuild resolves each row into a `NodeSpec` on the - background worker — tag, label, font, theme, handler, open state — and the main thread creates the - widgets from those specs, spread across frames. -7. **What a browser narrows to is its own.** Both tabs render one model, so which rows a browser shows - is decided by the panel showing it: a search typed in one tab leaves the other reading as it was, - and each browser opens in the mode a session left it in. -8. **The reader's shape is theirs to keep.** Which rows stand open is what the reader made of the - tree, so a browser records it and brings it back: a refresh, a change of filter and a repaint leave - the tree standing as it was, and so does the next run of the application. What a filter unfolds on - top of that shape is remembered as the filter's own, held for as long as the filter is, and handed - back when it goes — apart from a row the reader's own has come to stand on, which stays open so the - view they built stays on the screen. +1. **One reading of the disk feeds every view.** A refresh walks the reconstructions directory once into a `ReconstructionScan`, and every branch is built from that record. The views therefore agree about what exists by construction, and a folder name is parsed into its configuration fields once per refresh. +2. **The model carries the shape, and the panel carries the widgets.** Which rows exist, what they are called, which of them fold together and in what order they sit are decided on the tree. Both tabs render one model, so they show one shape, and each rule can be exercised without a window. +3. **A row's identity is its path, and its name is a label.** Favorites, the context menus, copy-path, playback and opening a reconstruction all key on `filepath`. That frees a name to be rewritten: a configuration directory renamed to its channel abbreviation, a chain of headings joined into one row, a colliding label marked with its configuration hash. +4. **The browser writes the headings the disk implies.** A frequency pair, a transformation, a source folder and one source audio each become a row that has no path of its own. What such a row offers follows from the subtree beneath it. +5. **One thing may stand in several places.** A reconstruction is listed by the configuration that produced it and again by the audio it was made from. An action on the thing, and not on the row, asks for every row representing it (`Tree.find_nodes`, `BrowserManager.nodes_at`) and hands them to both tabs. +6. **Per-row work happens off the main thread.** A rebuild resolves each row into a `NodeSpec` on the background worker (tag, label, font, theme, handler, open state), and the main thread creates the widgets from those specs, spread across frames. +7. **What a browser narrows to is its own.** Both tabs render one model, so the panel showing a browser decides which rows it shows. A search typed in one tab leaves the other reading as it was, and each browser opens in the mode a session left it in. +8. **The reader's shape is theirs to keep.** Which rows are open is what the reader made of the tree, so a browser records it and brings it back. A refresh, a change of filter and a repaint leave the tree standing as it was, and so does the next run of the application. What a filter unfolds on top of that shape is remembered as the filter's own. It is held for as long as the filter is and handed back when the filter goes. The one exception is a row the reader's own opening has come to stand on, which stays open so the view they built stays on the screen. --- -## The pipeline - -`BrowserManager` (`logic/reconstruction/browser/manager.py`) owns the tree and runs a refresh in four -steps: **scan** the directory, **build** each branch from that one scan, **shape** what came out, and -**publish** it through `Tree.set_root`. `BrowserLogic` sits above it as the surface the coordinators -drive, and `get_all_reconstruction_files` reads the scan. - -| Stage | Module | What it does | -|---|---|---| -| Scan | `tree/scan.py` | `scan_reconstructions` walks the directory once, recording each folder with the configuration its name states and each `.stn` file beneath it | -| Records | `tree/entries/` | `DirectoryEntry`, `ReconstructionEntry`, `ReconstructionScan` — frozen, path-only, no widgets and no tree | -| Configuration branch | `tree/configurations/` | `branch.py` lays the scanned folders out as they sit; `grouping.py` lifts a top-level configuration directory under frequency ▶ transformation configuration headings and names it by its channels, so the rows leading to it spell its display name; `naming.py` gives the remaining configuration directories friendly names, unique among their siblings | -| Sample branch | `tree/samples/` | `variants.py` regroups every top-level configuration directory's reconstructions by the audio they mirror (`SampleSource` → `SampleVariant`); `branch.py` rebuilds the mirrored folders as groups and gathers each audio's variants under one sample row, each labeled by its configuration | -| Shaping | `tree/prune.py`, `tree/collapse.py`, `tree/order.py` | Run in that order over each branch, deepest rows first | -| Containers | `tree/containers.py` | `find_or_create_group`, `find_or_create_config_group` and `find_or_create_sample` extend the heading of that name a parent already holds; each heading is looked up among the siblings of its own kind and class, so a folder and an audio sharing a name stay two rows | - -The policy the two branches share: a configuration directory sitting at the top level of the -reconstructions directory is the one lifted under groups and transposed into the sample view. A -configuration directory nested inside a plain folder keeps its friendly name where it sits, and a -reconstruction outside every configuration directory appears in the configuration branch, that being -the branch which follows the disk. - -## The node vocabulary - -`sampletones_core/structures/tree/` holds the nodes, all anytree-backed: - -* `TreeNode(name, node_type)` — a row and its kind. `NodeType.ROOT` for the container both branches - hang from, `GROUP` and `SAMPLE` for the headings the browser writes, `DIRECTORY` and `FILE` for what - the disk holds. -* `FileSystemNode(filepath)` — a row standing for a path. Favorites, playability, themes and the path - items all test for this class. -* `ConfigNode(config)` — a filesystem row belonging to a reconstruction configuration, carrying the - parsed `ConfigDirectoryFields`. It subclasses `FileSystemNode` so every reader of a path keeps - working, and the fields travel with the row, which is what lets a label, a tooltip and a font state - the configuration from the node already in hand. -* `ConfigGroupNode` — a heading gathering the configurations that share a stretch of their display - name: the rates they run at, the spectrum they were built from. It keeps `NodeType.GROUP`, so it - folds, prunes, sorts and behaves as any heading does, and it names the rows whose labels are - configuration text rather than words, which is what the configuration font reads. - -`create_directory_node` chooses between `FileSystemNode` and `ConfigNode` from the fields the scan -read. Which row carries the configuration follows the branch: in the configuration branch it is the -directory that names it, and in the sample branch it is the variant leaf, since there the -configuration is what distinguishes one row from the next. +## The two branches + +A refresh walks the directory once, and both branches are built from that one scan. The configuration branch lays the scanned folders out as they sit, giving each configuration directory a friendly name. The sample branch regroups every top-level configuration directory's reconstructions by the audio they mirror, and gathers each audio's variants under one sample row, each labeled by its configuration. A heading is extended and not repeated: one of that name is looked up among the siblings of its own kind and class, so a folder and an audio sharing a name stay two rows. + +The two branches share a policy. A configuration directory at the top level of the reconstructions directory is the one lifted under groups and transposed into the sample view. A configuration directory nested inside a plain folder keeps its friendly name where it sits. A reconstruction outside every configuration directory appears in the configuration branch, since that branch follows the disk. + +Which row carries the configuration follows the branch. In the configuration branch it is the directory that names it. In the sample branch it is the variant leaf, since there the configuration is what distinguishes one row from the next. ## The shaping rules -* **Prune** (`prune_empty_containers`) — a heading the browser wrote that gathers nothing leaves, - deepest first, so a whole chain of them goes at once and a reconstructions directory with nothing to - show stays silent. A folder the disk holds stays, since the configuration branch mirrors the disk. -* **Collapse** (`collapse_single_child_containers`) — a heading standing above a single row folds into - that row, which takes the joined name (`DISPLAY_SEPARATOR` between levels) and rises into its place. - The surviving row keeps its node type, path, configuration and children, so its click behavior, - theme, context menu and favorite star carry over. A fold that would repeat a name already beside it - stays open instead, and the branch roots stay in place. With a single configuration present the - configuration branch reads as one row per reconstruction, and it grows back into groups as soon as a - second configuration arrives. -* **Order** (`order_children`) — containers ahead of leaves, then `natural_sort_key` over the label, so - a row sits where its displayed name puts it and `8 kHz` precedes `44.1 kHz`. The pass runs once every - label is final; the branches directly under the container root keep the order the builder states them - in. -* **Unique sibling labels** (`unique_display_names`, `sampletones_core/configs/display.py`) — where - siblings would read alike, every member of that label takes its short configuration hash. One rule - serves the channel directories under a transformation group, the nested configuration directories, - and the variants under a sample. - -## The panels - -The browsers form one line of inheritance, each level owning what it shares: - -* `GUITreePanel` (`ui/elements/tree/tree.py`) — a tree of rows: the controls it narrows by and the - filter they compose, the shape it holds across rebuilds — kept for it by `RowExpansionMemory` - (`ui/elements/tree/expansion.py`) — the rebuild handshake, spec collection, themes and fonts per row, - the detail tooltip, the status-bar messages, and the context-menu items every browser can offer. -* `GUIFileBrowserPanel` (`ui/elements/tree/browser.py`) — a browser of files as a collapsible card: the - controls bringing the tree up to date and folding it away, the tree window, the folder-and-file - handler pair, and enabling the card as the tree locks and unlocks. A subclass declares its widgets as - a `FileBrowserTags` class attribute and states what its card and refresh control read. -* `GUIReconstructionBrowserPanel` (`ui/panels/shared/browser.py`) — the reconstruction browser: the - rows the two branches hold, the color a group and a sample read in, and the context menus. The - Reconstructions and Sequencer panels below it name their widgets, their refresh control, and what - opening a reconstruction means in that tab. - -The Main tab's filesystem explorer and the Instructions tab's library catalog sit on -`GUIFileBrowserPanel` as well, so the card, the search and the rebuild machinery are shared with them. - -**A rebuild** starts on the tree worker: `_launch_rebuild` takes the tree lock, brings the model up to -date, collects the rows into specs, and hands them to `TreeEmitter`, which clears the old rows and -stages the new ones in budget-sized batches so interactive callbacks run between slices. The -completion callback shows the empty state where one is called for, runs the panel's hook, and releases -the lock. Because a browser is asked to rebuild from either tab and from several places in the -application, exactly one rebuild is in flight at a time. A whole-tree rebuild asked for while the lock -is held — by another rebuild, a library load or a library generation — is kept for the release, which -asks for it again once the tree stands free; the latest request is the one kept. - -**A row's tag** (`compose_node_tag`, `ui/elements/tree/tag.py`) joins the names above it, which reads -the row back to whoever inspects the widget tree, and appends a digest over the exact path of -`(node_type, name)` pairs. Rows the names alone spell alike — a folder and the audio beside it, two -labels differing only in spacing or case — therefore keep tags of their own. A tag is composed rather -than stored, so any holder of a node can address its row: this is how expanding a subtree, repainting a -star and applying a filter reach the widgets. - -**What a row answers** follows its kind. A reconstruction plays on a click, opens on a double click, -and offers its path items, the tab's own actions and the favorite mark. A directory offers its path -items and the favorite mark. A group or a sample stands for no path, so its menu reads the subtree: how -many reconstructions it gathers, expanding and collapsing everything below it, the label the tree shows -it by, and — on a sample — the audio its reconstructions were made from, answered through any one of -them. - -**Favorites are paths.** `TreeLogic.is_node_favorite` tests the row's path against the session's set, -and `has_favorite_ancestor` tests the path's parents, so a reconstruction reads as part of a favorite -folder wherever a view puts it — including the sample branch, whose headings carry no path. Since one -path reaches the panel as several rows, `application.py` resolves the toggled path into every row -standing for it and hands them to both tabs, and each row repaints with the ancestry its own path -carries. +* **Prune.** A heading the browser wrote that gathers nothing leaves, deepest first, so a whole chain of them goes at once and a reconstructions directory with nothing to show stays silent. A folder the disk holds stays, since the configuration branch mirrors the disk. +* **Collapse.** A heading above a single row folds into that row, which takes the joined name and rises into its place. The surviving row keeps its node type, path, configuration and children, so its click behavior, theme, context menu and favorite star carry over. A fold that would repeat a name already beside it stays open, and the branch roots stay in place. With a single configuration present, the configuration branch reads as one row per reconstruction, and it grows back into groups as soon as a second configuration arrives. +* **Order.** Containers come ahead of leaves, then rows follow a natural sort over the label, so a row sits where its displayed name puts it and `8 kHz` precedes `44.1 kHz`. The pass runs once every label is final. The branches directly under the container root keep the order the builder gives them. +* **Unique sibling labels.** Where siblings would read alike, every member of that label takes its short configuration hash. One rule serves the channel directories under a transformation group, the nested configuration directories, and the variants under a sample. + +## Rebuilds and rows + +**One rebuild is in flight at a time.** A browser is asked to rebuild from either tab and from several places in the application. A rebuild starts on the tree worker, which takes the tree lock, brings the model up to date, collects the rows into specs and hands them to the emitter. The emitter clears the old rows and stages the new ones in budget-sized batches, so interactive callbacks run between slices. A whole-tree rebuild asked for while the lock is held, by another rebuild, a library load or a library generation, is kept for the release. The release asks for it again once the tree stands free, and the latest request is the one kept. + +**A row's tag is composed and not stored.** The tag joins the names above the row, which reads the row back to whoever inspects the widget tree, and it appends a digest over the exact path of `(node_type, name)` pairs. Rows the names alone spell alike, such as a folder and the audio beside it or two labels that differ only in spacing or case, therefore keep tags of their own. Since a tag is composed, any holder of a node can address its row. That is how expanding a subtree, repainting a star and applying a filter reach the widgets. + +**What a row answers follows its kind.** A reconstruction plays on a click, opens on a double click, and offers its path items, the tab's own actions and the favorite mark. A directory offers its path items and the favorite mark. A group or a sample represents no path, so its menu reads the subtree: how many reconstructions it gathers, expanding and collapsing everything below it, the label the tree shows it by, and, on a sample, the audio its reconstructions were made from, answered through any one of them. + +**Favorites are paths.** A row is a favorite when its path is in the session's set, and it reads as part of a favorite folder when any of its parents is. A reconstruction therefore reads as part of a favorite folder wherever a view puts it, including the sample branch, whose headings carry no path. One path reaches the panel as several rows, so `application.py` resolves the toggled path into every row representing it and hands them to both tabs. Each row repaints with the ancestry its own path carries. ## Filtering -`TreeFilter` (`ui/elements/tree/filter.py`) holds what a browser is currently asked to show, and the -panel showing it owns the filter. It is stated whole and replaced whole — `with_query`, -`with_favorites_only` — so one place resolves what the browser shows, and `NO_FILTER` is the filter a -browser showing its whole tree holds. +`TreeFilter` holds what a browser is currently asked to show, and the panel showing it owns the filter. It is given whole and replaced whole (`with_query`, `with_favorites_only`), so one place resolves what the browser shows. `NO_FILTER` is the filter a browser showing its whole tree holds. The two criteria answer different questions, so each lands in a different place: | Criterion | What it decides | Where it lands | What a change costs | |---|---|---|---| -| `favorites_only` | which rows the browser **draws** | `_append_spec` records the rows the mode shows, so `TreeEmitter` creates widgets for those alone | `redraw_tree` collects the rows again from the model in hand, on the tree worker | -| `query` | which of the drawn rows are **shown** | `update_tree_visibility` flips `show` over the rows already on screen, once the typing settles | a resolution of the query, debounced | - -One rule serves both. `TreeVisibility` (`sampletones_core/structures/tree/visibility.py`) takes the -rows a criterion named and answers which rows stay: a named row, a row leading down to one, and a row -one holds. `resolve_visibility` keeps the named rows and the rows above them, so what a pass holds in -memory follows the size of what was found, and a row beneath a match is answered from its own path -upward. - -**What a criterion names and what it keeps are two sets.** A criterion points the reader at some rows -and brings others along with them, and only the first kind is worth unfolding to. The rows a criterion -names are its **anchors**: for a search, the rows whose label matched; for the favorites mode, a row a -star sits on, and — where no row stands for the starred path — the shallowest rows that path reaches. -In the sample branch the headings carry no path, which is what makes the variants the rows a starred -folder arrives at. - -**A criterion is read the way that criterion means.** A search shows what a matching row gathers, so a -match opens along with the rows above it (`TreeVisibility.should_expand`). The favorites mode points -the reader at a star, so what opens is the rows above it (`_way_down_to`, over the anchors' ancestors) -while the star's own row stands where the reader left it — a starred folder is revealed. A starred -reconstruction inside a starred folder anchors on its own, which is what opens the folder above it. - -**Which stars are followed is the reader's.** The mode decides what is drawn; whether it also unfolds -is a preference stated per kind of favorite, held in `ApplicationConfig.browser` and offered as -**View ▸ Auto-expand favorites**. A starred reconstruction reads the reconstructions answer; a starred -folder, and everything it brings in where no row stands for it, reads the directories answer. Both are -off by default, so turning the mode on narrows the tree and leaves every row standing as it was. The -panel reads the pair through `TreeLogicProtocol`, once per resolution. - -**The way down opens on the pass the reader asked for, and stands for as long as the mode does.** -Switching the mode on is the reader asking to be shown their favorites, so the pass that switch starts -is the one that follows a star: `_state_favorites_only` records the request and `_resolve_filter` spends -it, and the rows it opens are noted in the mode's own memory. Later passes read that memory, so a -refresh, a query or a star gained meanwhile leaves the reader looking at their favorites, while the -stars followed stay the ones the switch asked about. Every turn of the mode is a pass's to answer: the -pass that reads the mode off lets the memory go and those rows fold back. A change of preference asks -for nothing; it is answered the next time the reader asks for the mode, which keeps a menu click from -moving the tree the reader is working in. - -**The way down becomes the reader's once their own rows stand on it.** A reader looking at their -favorites opens rows of their own below the way the mode opened, so a row of the mode's holds theirs on -the screen. `_release_mode_rows` therefore reads the model on the pass that finds the mode off — on the -tree worker, beside the other walks a pass makes — and hands the memory the ways down to the reader's -rows; `RowExpansionMemory.release` keeps the rows of the mode's among them, which writes the way down -into the shape a session keeps. What is left held the mode's opening alone, and folds with it. - -**A row the favorites mode holds back holds nothing it would show.** A row it shows either stands on -the way to a starred row or sits beneath one, and each of those facts holds for every row above it — so -declining a row declines its subtree, and one decision covers it while the traversal walks on. - -**Two memories, each holding what one hand opened.** `RowExpansionMemory` owns both and the rules that -join them, holding each row by the tag it is addressed under so a later pass creates it open again. The -reader's rows hold what the reader did — a click, read a frame later once the row has answered it, and -the expansion items and the collapse control, which record what they set — and they are what a session -writes down. The mode's rows hold the way down it opened, and go when the mode does, so a narrowed -browser hands the tree back the way the reader had it, keeping the rows theirs now stand on. Folding a -row is the reader's word on it whichever hand opened it, so `remember` releases the mode's claim along -with the reader's and the row stays folded. A pass writes from the tree worker while a click writes from -the main thread, so one lock covers every answer the memory gives. Both sets are held to the rows the -model states, read afresh on every pass, so a row a moved reconstructions directory left behind leaves -them with it. Which browsers record a shape at all is `_REMEMBERS_EXPANSION`: it decides whether a click -is followed through to the memory, and a browser that keeps none leaves it empty. - -A search unfolds by the same rule from the other end: its matches and the rows above them open for as -long as the query stands, resolved afresh on each pass, and clearing the query folds them back. - -The shape outlives the run as well. A browser is handed the mode and the rows it opens with as it is -built (`initial_favorites_only`, `initial_expanded_rows`). A change of mode is written where it happens, -through `on_favorites_filter_changed`, and the shape is asked for the once, at exit: -`_persist_application_state` takes each tab's rows into `ApplicationState.expanded_rows` under the -panel's tag, so a pass holds what it opened in memory, on the tree worker, and the session file reads it -from there. - -**The Main tab's explorer remembers folders, not rows.** Its rows are the folders on disk, read a level -at a time as the reader opens one, so `ExplorerManager` holds two facts about a folder: whether its -children have been read, and whether its row stands open. They part company — a folder read and then -folded away is loaded and closed — and the open one is the shape a session writes to -`ApplicationState.expanded_directories`. A refresh reads down to each remembered folder through -`_expand_path_to`, reading every folder it needs once. A remembered folder the disk no longer holds is -read down to as far as it still stands and stays remembered, so a drive unplugged for one run opens -where it was left once it is back. - -**What the mode costs.** Resolving it walks the model once per rebuild, on the tree worker, testing each -row with `is_node_favorite` and `has_favorite_ancestor` — set lookups over `filepath.parents` — and the -anchors the preference follows are read out of that one answer. What it materializes is the starred rows -and the rows above them, and what reaches DearPyGui is the drawn rows alone: on a directory holding -hundreds of thousands of reconstructions, a favorites-only browser creates widgets for the starred ones -and their headings. A keystroke resolves the query alone, the drawn rows being the mode's to state. A -favorite toggled while the mode is on redraws the browser, so starring a row brings it in and unstarring -one takes it out along with what it held. - -A rebuild that drew no row fills the cleared tree with the message naming the criterion that came back -empty (`global.dialog.message.tree_no_favorites`, `global.dialog.message.tree_no_results`), so the -filter's answer reads where the rows would be. - -**The control** is a checkbox under the search box carrying the favorite glyph, which reads in the -favorite color while the mode is on and muted while it is off. `_OFFERS_FAVORITES_FILTER` states -which cards hold it: the reconstruction browsers, whose rows stand for the paths a session stars. It -follows the tree's lock, a rebuild being what it asks for, and its label reads in the pair every -checkbox reads — the text color while it can be clicked, the muted one while a rebuild holds it — so -the shade states whether the control is live. - -Each browser opens in the mode it was left in. The panel raises `on_favorites_filter_changed` with its -own tag, and the tab coordinator writes it to `ApplicationState.favorites_filters` under that tag, -which is how a collapsed card is remembered too. - -**Folding the whole tree away** is the other control every card carries. It reaches the rows through the -model rather than the widget tree, so one pass covers a branch however deep it runs, and it records what -it set — leaving the memory empty, which is the shape a later pass then draws. The explorer folds first -and drops the folders it had read afterward, so opening one lists it as it stands on disk. +| `favorites_only` | which rows the browser **draws** | the rows the mode shows are the rows collected into specs, so `TreeEmitter` creates widgets for those alone | a redraw: the rows are collected again from the model in hand, on the tree worker | +| `query` | which of the drawn rows are **shown** | `show` is flipped over the rows already on screen, once the typing settles | a resolution of the query, debounced | + +One rule serves both. `TreeVisibility` takes the rows a criterion named and answers which rows stay: a named row, a row leading down to one, and a row one holds. It keeps the named rows and the rows above them, so what a pass holds in memory follows the size of what was found, and a row beneath a match is answered from its own path upward. + +**What a criterion names and what it keeps are two sets.** A criterion points the reader at some rows and brings others along with them, and only the first kind is worth unfolding to. The rows a criterion names are its **anchors**. For a search, the anchors are the rows whose label matched. For the favorites mode, an anchor is a row a star sits on, or, where no row represents the starred path, the shallowest rows that path reaches. In the sample branch the headings carry no path, so the variants are the rows a starred folder arrives at. + +**A criterion is read the way that criterion means.** A search shows what a matching row gathers, so a match opens along with the rows above it. The favorites mode points the reader at a star, so what opens is the rows above it, while the star's own row stands where the reader left it. A starred folder is therefore revealed. A starred reconstruction inside a starred folder anchors on its own, and that opens the folder above it. + +**Which stars are followed is the reader's.** The mode decides what is drawn. Whether it also unfolds is a preference given per kind of favorite, held in the application's browser configuration. A starred reconstruction reads the preference for reconstructions. A starred folder, and everything it brings in where no row represents it, reads the preference for directories. By default both are off, so turning the mode on narrows the tree and leaves every row standing as it was. The panel reads the pair through `TreeLogicProtocol`, once per resolution. + +**An opening belongs to the pass that asked for it.** Switching the mode on is the reader asking to be shown their favorites, so the pass that switch starts is the one that follows a star, and the rows it opens are noted in the mode's own memory. Later passes read that memory. A refresh, a query or a star gained meanwhile therefore leaves the reader looking at their favorites, while the stars followed stay the ones the switch asked about. The pass that reads the mode off lets the memory go, and those rows fold back. A change of preference asks for nothing. It is answered the next time the reader asks for the mode, which keeps a menu click from moving the tree the reader is working in. + +**A row the favorites mode holds back has nothing it would show.** A row the mode shows either stands on the way to a starred row or sits beneath one, and each of those facts holds for every row above it. Declining a row therefore declines its subtree, and one decision covers it while the traversal walks on. + +**Two memories, each holding what one hand opened.** `RowExpansionMemory` (`ui/elements/tree/expansion.py`) owns both and the rules that join them. It holds each row by the tag it is addressed under, so a later pass creates the row open again. + +- The reader's rows hold what the reader did: a click, the expansion items, the collapse control. A session writes them down. +- The mode's rows hold the way down it opened, and they go when the mode does. A narrowed browser therefore hands the tree back the way the reader had it. + +Where the reader's own rows have come to stand on a row of the mode's, the memory takes that way down over as the reader's, so the view they built stays on the screen (principle 8). Folding a row is the reader's word on it whichever hand opened it, so the row stays folded. A pass writes from the tree worker while a click writes from the main thread, so one lock covers every answer the memory gives. Both sets are held to the rows the model states, read afresh on every pass, so a row a moved reconstructions directory left behind leaves them with it. Each browser is configured to record a shape or not, and a browser that records none leaves the memory empty. + +A search unfolds by the same rule from the other end: its matches and the rows above them open for as long as the query stands, resolved afresh on each pass, and clearing the query folds them back. + +**The shape outlives the run.** A browser is handed the mode and the rows it opens with as it is built, reports a change of mode where it happens, and is asked for its shape once, at exit. Each tab's rows go to `ApplicationState.expanded_rows` under the panel's tag, and its mode to `ApplicationState.favorites_filters` under the same tag. A browser therefore opens in the mode it was left in, with the rows it was left with, and a collapsed card is remembered the same way. + +**The Main tab's explorer remembers folders.** Its rows are the folders on disk, read a level at a time as the reader opens one. `ExplorerManager` therefore holds two facts about a folder: whether its children have been read, and whether its row is open. They part company, since a folder read and then folded away is loaded and closed. The open one is the shape a session writes to `ApplicationState.expanded_directories`. A refresh reads down to each remembered folder, reading every folder it needs once. A remembered folder the disk no longer holds is read down to as far as it still stands and stays remembered, so a drive unplugged for one run opens where it was left once it is back. + +**What the mode costs.** Resolving the mode walks the model once per rebuild, on the tree worker, testing each row's path and its parents against the session's set. The anchors the preference follows are read out of that one answer. The resolution materializes the starred rows and the rows above them, and what reaches DearPyGui is the drawn rows alone. A favorites-only browser therefore creates widgets for the starred reconstructions and their headings and not for the whole tree, even over a very large directory. A keystroke resolves only the query, because the mode decides the drawn rows. A favorite toggled while the mode is on redraws the browser, so starring a row brings it in and unstarring one takes it out along with what it held. + +A rebuild that drew no row fills the cleared tree with the message naming the criterion that came back empty, so the filter's answer reads where the rows would be. + +**The controls each card carries.** A checkbox under the search box switches the favorites mode. Only the reconstruction browsers hold it, because their rows represent the paths a session stars, and it follows the tree's lock, since a rebuild is what it asks for. Folding the whole tree away is the other control. It reaches the rows through the model and not the widget tree, so one pass covers a branch however deep it runs, and it records what it set, which leaves the memory empty. The explorer folds first and drops the folders it had read afterward, so opening one lists it as it stands on disk. diff --git a/docs/development/application/config-organization.md b/docs/development/application/config-organization.md index 94cbd1d29..57b37ac45 100644 --- a/docs/development/application/config-organization.md +++ b/docs/development/application/config-organization.md @@ -1,23 +1,12 @@ # Configuration Organization -_SampleToNES_ ships its configuration as a YAML data package, `sampletones_config`. This -document states the principles that decide where a configuration value belongs and how it -is read; use it as the reference when adding or moving a value. It sits alongside -`docs/development/architecture.md` (application layering and ownership) and -`docs/development/guidelines.md` (coding rules). - -"Configuration" names three separate things in this codebase. This document governs the -first: - -- **Shipped configuration** — the `sampletones_config` YAML package: layout, theme, - palettes, keybindings, language, behavior, deployment, and the import boundaries. - *(This document.)* -- **Runtime user preferences** — mutable state persisted to the user profile - (`sampletones_application/config`, e.g. `PlaybackConfig`, `ShortcutsConfig`, - `ApplicationState`), governed by that package. -- **Project generation settings** — JSON stored beside a project - (`sampletones_core/configs`, `config.json`), documented in - `docs/formats/configuration.md`. +_SampleToNES_ ships its configuration as a YAML data package, `sampletones_config`. This document sets out the principles that decide where a configuration value belongs and how it is read. Use it as the reference when adding or moving a value. It sits alongside [`architecture.md`](../architecture.md) (application layering and ownership) and [`guidelines.md`](../guidelines.md) (coding rules). + +The word "configuration" names three different things in this codebase. This document governs the first: + +- **Shipped configuration**: the `sampletones_config` YAML package with layout, theme, palettes, keybindings, language, behavior, deployment and the import boundaries. *(This document.)* +- **Runtime user preferences**: mutable state persisted to the user profile (`sampletones_application/config`, for example `PlaybackConfig`, `ShortcutsConfig`, `ApplicationState`), governed by that package. +- **Project generation settings**: JSON stored beside a project (`sampletones_core/configs`, `config.json`), documented in [the configuration file format](../../formats/configuration.md). --- @@ -25,162 +14,59 @@ first: ### 1. Data and meaning are separate -`sampletones_config` carries values; the schema that interprets them lives in the package -that reads them. The dependency runs one way — a consumer imports the data package only to -resolve its directory (`CONFIG_DIRECTORY`), and the package itself is pure YAML with an -empty `__init__.py`. Each schema lives with its reader: +`sampletones_config` carries values, and the schema that interprets them lives in the package that reads them. The dependency runs one way. A consumer imports the data package only to resolve its directory (`CONFIG_DIRECTORY`), and the package itself is pure YAML with an empty `__init__.py`. Each schema lives with its reader: -- `sampletones_application` owns the layout, theme, palettes, keybindings, language, - behavior, and deployment schemas. +- `sampletones_application` owns the layout, theme, palette, keybinding, language, behavior and deployment schemas. - `sampletones_tools` owns the import-boundary schemas. - `sampletones_shared` owns the loader primitives (`load_yaml_model`, `load_yaml_model_dir`). -So the data carries the values and the consumer carries the meaning, and the two evolve -on their own terms. +The data therefore carries the values and the consumer carries the meaning, and the two evolve on their own terms. -Data a package reads ships with that package, beside the schema reading it: the matching -defaults every reconstruction starts from sit in `sampletones_core/configs/generation.yaml`, -the calibration tuning in `sampletones_tools/calibration/config/`, the synthetic corpus in -`sampletones_tools/corpus/config/` and the application mark in `sampletones_tools/assets/mark/config/`, -each placed by `package_directory`. `sampletones_config` -holds what the application reads and the import boundaries, which state the repository's -package layers for every tree and which [package layers](../packages.md) refers to. +Data a package reads ships with that package, beside the schema that reads it, and `package_directory` places it. For example, the defaults every reconstruction starts from sit in `sampletones_core/configs/generation.yaml`. `sampletones_config` holds what the application reads and the import boundaries. The boundaries state the repository's package layers for every tree, and [package layers](../packages.md) refers to them. ### 2. The top level is organized by domain -`sampletones_config` has one top-level directory per schema family and its loader: -`application`, `behavior`, `boundaries`, `keybindings`, `lang`, `layout`, -`palettes`, `theme`. Each domain owns its schema and its load path (see -[Domains](#domains)). A new domain is a new top-level directory with its own schema owner -and loader. - -Palettes are a domain of their own because two other domains resolve against them: a color -field in `layout/` and a color entry in `theme/` both name a palette token, and the palette -is what turns that name into a value. A directory holds one file per palette, named after the -palette it declares, and every palette answers the same token set — an entry names one token -and each palette must have an answer for it. - -Keybindings are a domain on the same shape: a scheme is a named set a preference selects by -name, so the directory holds one file per scheme, named after the scheme it declares, and -every scheme answers the same action set — an entry names one `ShortcutId` and each scheme -must have a combination for it. What the directory carries is the combinations, which are a -reader's to choose; the actions and the category each belongs to are code, since they follow -from the scope that handles the press. +`sampletones_config` has one top-level directory per schema family and its loader. Each domain owns its schema and its load path (see [Domains](#domains)). A new domain is a new top-level directory with its own schema owner and loader. + +Palettes are a domain of their own because two other domains resolve against them. A color field in `layout/` and a color entry in `theme/` both name a palette token, and the palette turns that name into a value. A directory has one file per palette, named after the palette it declares, and every palette answers the same token set: an entry names one token, and each palette must have an answer for it. + +Keybindings are a domain of the same shape. A scheme is a named set that a preference selects by name, so the directory has one file per scheme, named after the scheme it declares, and every scheme answers the same action set. An entry names one `ShortcutId`, and each scheme must have a combination for it. The directory carries the combinations, which are a reader's to choose. The actions and the categories are code ([`keyboard.md`](keyboard.md)). ### 3. The config tree mirrors the code -The layout config is shaped like the code that reads it: its directory tree matches the -`LayoutConfig` model tree, which mirrors the application's feature-area taxonomy across -`ui/panels/`, `logic/`, and `view_model/`. A value's place in the config therefore -predicts its place in the code. Three conventions keep the mirror true: - -- **A feature area is a directory of fragments.** Each area is a directory loaded by - `load_yaml_model_dir`; every `.yaml` supplies the model's ``, and an - optional `root.yaml` carries the loose scalars that own no section file. Two - cross-cutting resources — `fonts.yaml` and `glyphs.yaml` — are single self-contained - files at the `layout/` root, each one resource in one file. -- **File stem = field = model.** `choice.yaml` fills field `choice`, validated by - `ChoiceLayout` in `choice.py`; the three names match within a domain, so one name traces - a value from YAML through field to schema. A stem is unique within its domain: the same - name may recur across domains as a related-but-distinct resource - (`layout/general/plus_minus_buttons.yaml` sizes a widget while - `theme/plus_minus_buttons.yaml` styles it — one widget, two domains, loaded separately). -- **Tabs sit where their coordinators sit.** The four notebook tabs live under - `layout/tabs/` (`main`, `instructions`, `reconstruction`, `sequencer`), matching - `coordinators/tabs/` and aggregated as `LayoutConfig.tabs`; a tab's configuration is - found where its code is. Cross-tab and shared areas stay at the `layout/` root as their - own feature areas: `general/`, the plot-element family `graphs/`, the transport toolbar - `player/`, and the dialogs `project_properties/` and `settings/`. +The layout config is shaped like the code that reads it. Its directory tree matches the `LayoutConfig` model tree, which mirrors the application's feature-area taxonomy across `ui/panels/`, `logic/` and `view_model/`. A value's place in the config therefore predicts its place in the code. Three conventions keep the mirror true: + +- **A feature area is a directory of fragments.** `load_yaml_model_dir` loads each area. Every `.yaml` supplies the model's ``, and an optional `root.yaml` carries the loose scalars that have no section file. Two cross-cutting resources, `fonts.yaml` and `glyphs.yaml`, are single self-contained files at the `layout/` root. +- **File stem = field = model.** `choice.yaml` fills field `choice`, validated by `ChoiceLayout` in `choice.py`. The three names match within a domain, so one name traces a value from YAML through field to schema. A stem is unique within its domain. The same name may recur across domains as a related but distinct resource: `layout/general/plus_minus_buttons.yaml` sizes a widget, and `theme/plus_minus_buttons.yaml` styles it. +- **Tabs sit where their coordinators sit.** The notebook tabs live under `layout/tabs/`, matching `coordinators/tabs/` and aggregated as `LayoutConfig.tabs`, so a tab's configuration is found where its code is. Cross-tab and shared areas stay at the `layout/` root as their own feature areas: `general/`, the plot-element family `graphs/`, the transport toolbar `player/`, and the dialogs `project_properties/` and `settings/`. ### 4. Every value has one home -A value lives in exactly one place, owned by the concept it describes. A tab's own -geometry — width and height together — lives in that tab's directory. The shared outer -column skeleton (`side`, `center_weight`) lives in `general/`, since it belongs to every -tab equally. The values that drive responsive resizing (`baseline_viewport_width`, -`baseline_viewport_height`, the graph-stack cap) live together in `general/responsive.yaml` -because they are the input to one scheme. Ownership decides the home: whoever owns the -concept holds the value, and it appears once. +A value lives in exactly one place, owned by the concept it describes. A tab's own geometry, width and height together, lives in that tab's directory. The shared outer column skeleton (`side`, `center_weight`) lives in `general/`, since it belongs to every tab equally. The values that drive responsive resizing (`baseline_viewport_width`, `baseline_viewport_height` and the graph-stack cap) live together in `general/responsive.yaml`, because they are the input to one scheme. Ownership decides the home: whoever owns the concept holds the value, and it appears once. ### 5. Storage shape and consumer shape differ -Principles 1–4 shape the config for **where a value is authored**. What a consumer needs -is often a different shape, and the two are reconciled at the composition root -(`Application.__init__`, where the validated `LayoutConfig` already exists). Each consumer -receives a view built for it: - -- A **feature consumer** — a panel or UI element — receives its model whole: - `GUIConverterPanel(layout: ConverterLayout)`. The model carries all of that panel's own - geometry, so a new field reaches the panel through the model it already holds. -- A **generic primitive** — the responsive math (`expanded_side_width`, - `stacked_graph_height`), the column builders (`ColumnSpec`, `TabColumns`), raw - `dpg.configure_item` — receives the plain integers it computes with, since it blends a - config value with a live runtime measurement (e.g. `dpg.get_viewport_client_width()`) - and works below the level of any layout model. -- A **tab coordinator** receives a per-tab view. A frozen-dataclass DTO in the - `parameters/` package gathers exactly what one tab needs: the shared six-field - `TabGeometry` core, the flat integers its primitive sinks consume, and the cohesive - feature models it forwards whole (`SchedulingBehavior`, `GraphsLayout`, the tab's own - `Layout`, the color blocks). A small factory produces any narrowed slice a consumer - needs (`TreeColors.create`, `PitchStepperStyle.from_general`). - -The type signals which side of the boundary a value is on: a frozen Pydantic model with -`extra="forbid"` is a YAML fragment; a `@dataclass(frozen=True)` is a view derived in code. -The composition root is the one place that knows both shapes, so each deep path from -storage to consumer is written once, in one factory. This is what the DTO layer is for: a -coordinator depends only on the narrowed view handed to it, and the knowledge of where -each value sits in the tree stays in the factory. +Principles 1–4 shape the config for **where a value is authored**. A consumer often needs a different shape. The composition root (`Application.__init__`, where the validated `LayoutConfig` already exists) reconciles the two and hands each consumer a view built for it: + +- A **feature consumer**, a panel or UI element, receives its model whole: `GUIConverterPanel(layout: ConverterLayout)`. The model carries all of that panel's own geometry, so a new field reaches the panel through the model it already holds. +- A **generic primitive**, such as the responsive math, the column builders or raw `dpg.configure_item`, receives the plain integers it computes with. It blends a config value with a live runtime measurement (for example `dpg.get_viewport_client_width()`) and works below the level of any layout model. +- A **tab coordinator** receives a per-tab view: a frozen-dataclass DTO in the `parameters/` package that gathers exactly what one tab needs. A small factory produces any narrowed slice a consumer needs (`TreeColors.create`, `PitchStepperStyle.from_general`). + +The type shows which side of the boundary a value is on. A frozen Pydantic model with `extra="forbid"` is a YAML fragment, and a `@dataclass(frozen=True)` is a view derived in code. The composition root is the one place that knows both shapes, so each deep path from storage to consumer is written once, in one factory. A coordinator depends only on the narrowed view handed to it, and the knowledge of where each value sits in the tree stays in the factory. --- ## Domains -| Domain | Directory | Schema owner | How it is loaded | -|--------|-----------|--------------|------------------| -| Application | `application/` | `DeploymentConfig` (`sampletones_application/config/deployment/`) | `DeploymentConfig.load()`, with `SAMPLETONES_*` env overrides | -| Behavior | `behavior/` | `BehaviorConfig` (`sampletones_application/layout/behavior.py`) | folded into `LayoutConfig.behavior` by `load_layout_config` | -| Boundaries | `boundaries/` | `ImportBoundaryRules` (`sampletones_tools/checks/boundary/configs/`) | `ImportBoundaryRules.load()` | -| Keybindings | `keybindings/` | `ShortcutScheme` (`sampletones_application/utils/gui/shortcuts/`) | `ShortcutCatalog.load()`, indexed by scheme name | -| Language | `lang/` | `LanguageManager` (`sampletones_application/categories/`) | flat string map keyed `page.panel.text_type.element`, each key validated at load | -| Layout | `layout/` | `LayoutConfig` (`sampletones_application/layout/config.py`) | `load_layout_config` (`layout/loader.py`) | -| Palettes | `palettes/` | `Palette` (`sampletones_application/utils/palette/`) | `PaletteCatalog.load()`, indexed by palette name | -| Theme | `theme/` | `ThemeSpec` (`sampletones_application/ui/themes/spec.py`) | `ThemeLoader.load_all()` → `ThemeRegistry` | - -The palettes load first, and the source holding the active one is injected as validation -**context**, so any color field in layout or theme keeps the token it was written as and -reads its value from the palette in place when it is drawn with. `PaletteCatalog` names the -palette a preference selects and answers with the default (`studio`) for a name the build -does not ship, so a preference outlives the build that wrote it. - -`ShortcutCatalog` answers the same way for a keybinding scheme, with the shipped `default` as -its fallback. A scheme is validated as it is read: every action the application names is -answered, every key name resolves against the key table, and one combination reaches one -action within a category, so a scheme in use resolves any press its category owns. The user's -own rebindings stay on the preference side (`ShortcutsConfig`) and are applied over the -selected scheme at startup, which keeps the shipped file the statement of what a build offers. - -The domain holds one file per keyboard the build ships — `default.yaml` and `macos.yaml` — -and the platform decides which one a profile starts on: `ShortcutsConfig.scheme` takes its -default from `PLATFORM_SCHEME_NAMES`, so the choice is made when the configuration is created -and the name stored there selects the scheme on every run after. - -Layout and theme schemas are `frozen=True, extra="forbid"`, and loading is eager at the -composition root (`Application.__init__` → `load_layout_config`, wrapped as `SystemError`), -so a mismatch between YAML and schema surfaces loudly at startup. - -Behavior loads as its own domain — its own directory, schema owner (`BehaviorConfig`), and -`load_yaml_model` call — and attaches to the layout result as `LayoutConfig.behavior`, so -consumers reach it as `layout.behavior.*`. A single access path serves the ~15 runtime -sites across `application.py` and the tab coordinators that read it, and the ~10 modules -that import `SchedulingBehavior` as a type. - -Boundaries is the domain a developer tool reads. It states the layer graphs the packages -divide into, the imports each part of the application stays clear of, the spellings a tree -keeps out, and the standard-library rule the bootstrap scripts under `scripts/` hold to, and -`sampletones check import-boundary` runs it over the source and scripts trees on every commit. A declaration draws on the named prefix groups `general.yaml` holds, so a set several -rules reach for is written once and each rule names it, and a name reaching no group is -refused as the domain is read. The bundle carries the domain with the rest of -`sampletones_config`. +Each domain is one top-level directory with one schema owner and one load path. `sampletones_config/README.md` lists the directories and the schema owning each, beside the data itself. + +The palettes load first, and the source holding the active one is injected as validation **context**. Any color field in layout or theme therefore keeps the token it was written as and reads its value from the palette in place when it is drawn with. `PaletteCatalog` names the palette a preference selects and answers with the shipped default for a name the build does not carry, so a preference outlives the build that wrote it. `ShortcutCatalog` answers the same way for a keybinding scheme. [`keyboard.md`](keyboard.md) describes how a scheme is validated, layered under a reader's own rebindings and chosen per platform. + +Layout and theme schemas are `frozen=True, extra="forbid"`, and loading is eager at the composition root, wrapped as `SystemError`, so a mismatch between YAML and schema surfaces loudly at startup. + +Behavior loads as its own domain, with its own directory, schema owner and load call. It attaches to the layout result as `LayoutConfig.behavior`, so every consumer reaches it as `layout.behavior.*` through one access path. + +Boundaries is the domain a developer tool reads, not the application. It declares the layer graphs the packages divide into, the imports each part of the application stays clear of, the spellings a tree keeps out, and the standard-library rule the bootstrap scripts hold to. A declaration draws on the named prefix groups in `general.yaml`, so a set that several rules reach for is written once, and a name that reaches no group is refused as the domain is read. [Package layers](../packages.md) says what the graphs mean. --- @@ -188,24 +74,8 @@ refused as the domain is read. The bundle carries the domain with the rest of Three load mechanisms serve the three grouping schemes: -- **Field aggregation** (layout, and every domain that mirrors the code). - `load_layout_config` builds `LayoutConfig` field by field — `load_yaml_model` for a - single-mapping file, `load_yaml_model_dir` for a feature-area directory — so the - directory structure and the model structure stay identical (principle 3), validated - strictly with `extra="forbid"`. -- **Tag-graph discovery** (theme). Theme is organized for human navigation, grouped by the - widget family it styles (`button/`, `channels/`, `dialog/`, `header/`, `nodes/`, - `panel/`, `player/`, `tables/`). `ThemeLoader.load_all()` reads every `*.yaml` under - `theme/` recursively, validates each as a `ThemeSpec`, resolves the `extends` inheritance - graph, and registers the results in the `ThemeRegistry` singleton keyed by `tag`. Here - the directory grouping serves people and the `tag` and `extends` fields carry the load - meaning; every theme extends the base `default` unless it names another parent. -- **Name-keyed discovery** (palettes, keybindings). `PaletteCatalog.load()` reads every - `*.yaml` under `palettes/` and indexes it by `Palette.name`, holding each file's stem - against the name it declares so one name traces a palette from a stored preference to the - file on disk. `ShortcutCatalog.load()` reads `keybindings/` the same way, keyed by - `ShortcutScheme.name`. - -Deployment and the boundaries each load through a bespoke `.load()` -classmethod over the same low-level primitives in -`sampletones_shared/utils/serialization.py` — the one module that calls `yaml.safe_load`. +- **Field aggregation** (layout, and every domain that mirrors the code). The config is built field by field, as principle 3 describes, and validated strictly with `extra="forbid"`. +- **Tag-graph discovery** (theme). Theme is grouped by the widget family it styles, for a person navigating it. Each spec's `tag` and `extends` carry the load meaning: every file under `theme/` is read, validated, resolved against its parent and registered by tag. Every theme extends the base unless it names another parent. +- **Name-keyed discovery** (palettes, keybindings). Every file in the directory is read and indexed by the name it declares, with each file's stem held against that name, so one name traces a palette or a scheme from a stored preference to the file on disk. + +Deployment and the boundaries each load through a `.load()` classmethod of their own, over the same low-level primitives in `sampletones_shared/utils/serialization.py`, the one module that calls `yaml.safe_load`. diff --git a/docs/development/application/dialogs.md b/docs/development/application/dialogs.md new file mode 100644 index 000000000..93d6c90c2 --- /dev/null +++ b/docs/development/application/dialogs.md @@ -0,0 +1,60 @@ +# Dialogs + +A dialog sets its size once, and where it opens follows from that size. Consult this when a dialog opens at +the wrong size or in the wrong place, and when adding one. `GUIWindow` is the single place a dialog's +window is opened, so what this document says holds for every dialog the application raises. + +## What a dialog sets + +`DialogGeometry` (`layout/primitives.py`) carries a dialog's whole geometry: + +- **`width`** is always set and is held both ways: it is the least the window may take and the most. A + stretched item (a field, a combo, a button at `width=-1`) measures itself against the region the window + offers. A window free to widen to its content, with content sized from the window, hands each other a + little more every frame until the screen stops them. Holding the width at what the dialog sets leaves the + two agreeing from the first frame. It also gives every dialog the same reading width whatever it holds. +- **`height`** is the size the dialog opens at. A dialog holding more than that grows to hold it, so a + reader is always shown the whole of what the dialog says. A dialog that learns its length as it opens, + such as a prompt whose text wraps or a form that unfolds a group once a run begins, leaves the height out + and takes the height its content asks for. + +Both must be above zero, so a position can always be computed from a dialog's size. + +## Where a dialog opens + +Every dialog opens centered on the viewport's client area, which is the space a position is measured in. + +A dialog with a height is placed before it is ever drawn. Both numbers are in hand, so `GUIWindow.show` sets +the position between building the tree and the first frame that carries it. DearPyGui leaves an unplaced +modal where the pointer last was, and a position set through the API takes precedence over that. A placed +dialog therefore never appears at the pointer. + +A dialog whose height its content settles has nothing to place from until a frame has measured it, so the +correction centers it. It is drawn once where the modal opened, once centered against the height it had +reached by then, and it stands where it belongs from the third frame on. + +The correction re-reads the drawn size each frame and centers the window against it, so a dialog stays +centered all the way to the size it settles at. The correction ends once two readings agree. It ends on +purpose: a dialog has no `no_move`, so a reader can drag it, and a pass that kept measuring would drag it +back. The one window that changes size while it stands is the error dialog, whose **Show traceback** +unfolds a text box beneath the message. The correction has ended by then, so the dialog grows downward from +where it stands. + +Each axis is held at zero at the least, so a dialog taller than the viewport keeps its title bar reachable. + +## Where it is written + +`GUIWindow.dialog_window` (`ui/elements/window.py`) is the only place a dialog's `dpg.window` is opened. +Every dialog is therefore modal, resists resizing and collapse, and offers the title bar's close button +exactly where it answers for closing. `GUIDialogWindow` adds the keyboard ring over it. The windows under +`utils/gui/dialogs/windows/` are the shapes the application raises without writing a window of its own: a +confirmation, a save prompt, an error report and a notice. A dialog that belongs to one tab lives under +`ui/panels/dialogs/`. + +`centered_position` (`utils/placement.py`) is the arithmetic, and `viewport_center` and +`center_when_settled` (`utils/gui/align.py`) are the readings. `FrameCallbackManager` carries the frame the +correction waits on. [render-thread.md](render-thread.md) says why it counts frames and does not wait on +one. + +Who *raises* a dialog is a different question, and [architecture.md](../architecture.md) answers it: dialog +presentation belongs to coordinators. diff --git a/docs/development/application/keyboard.md b/docs/development/application/keyboard.md index 9dcbc6696..8ef52c06b 100644 --- a/docs/development/application/keyboard.md +++ b/docs/development/application/keyboard.md @@ -1,99 +1,58 @@ # The Keyboard and the Actions It Reaches -This document describes how a key press reaches behavior in `sampletones_application`, and how an -**action** — the one name a press, a menu item and a context item all reach one behavior by — is -declared and shown. It governs `utils/gui/keyboard/`, `utils/gui/shortcuts/`, the schemes under -`sampletones_config/keybindings/`, and the menu surfaces that print an action. Consult it when -giving a panel keys of its own, adding a shortcut, or putting an action on a menu. +This document describes how a key press reaches behavior in `sampletones_application`, and how an **action** is declared and shown. An action is the one name a press, a menu item and a context item all reach one behavior by. The document governs `utils/gui/keyboard/`, `utils/gui/shortcuts/`, the schemes under `sampletones_config/keybindings/`, and the menu surfaces that print an action. Consult it when giving a panel keys of its own, adding a shortcut, or putting an action on a menu. -The design truths it realizes are principles 12 and 14 of [`architecture.md`](../architecture.md): -one dispatcher owns the keyboard, and an action is declared once. This document holds the -mechanism behind both. +The design truths it realizes are principles 12 and 14 of [`architecture.md`](../architecture.md): one dispatcher owns the keyboard, and an action is declared once. This document describes the mechanism behind both. --- ## The dispatcher -DearPyGui delivers a press to every registered key handler with the same global reach, and gives -none of them a way to stop another — or ImGui itself — from also seeing it, so priority and consume -semantics exist only where the application builds them. A single `KeyRouter` -(`utils/gui/keyboard/`) owns the one `add_key_press_handler` for the whole application, snapshots -the modifier state once into a frozen `KeyEvent`, and offers that event to registered **scopes** -from highest priority to lowest. The first active scope whose handler returns `True` claims the -press and ends the walk; this software walk is the sole consume mechanism the framework -leaves available. +DearPyGui delivers a press to every registered key handler with the same global reach. No handler can stop another handler, or ImGui itself, from also seeing it. Priority and consume semantics therefore exist only where the application builds them. -Each keyboard consumer registers one scope through `register(handle, *, priority, active)`, where -`active()` reports whether the scope wants keys at this moment and `handle(event) -> bool` acts on -the press and reports whether it claimed it. +A single `KeyRouter` (`utils/gui/keyboard/`) owns the one `add_key_press_handler` for the whole application. It snapshots the modifier state once into a frozen `KeyEvent` and offers that event to registered **scopes** from the highest priority to the lowest. The first active scope whose handler returns `True` claims the press and ends the walk. This software walk is the only consume mechanism the framework leaves available. + +Each keyboard consumer registers one scope through `register(handle, *, priority, active)`. `active()` reports whether the scope wants keys at this moment, and `handle(event) -> bool` acts on the press and reports whether it claimed it. ### Priorities -Three priorities order the whole application: +The application orders its scopes by priority, from highest to lowest: | Priority | Scope | Active when | Behavior | |----------|-------|-------------|-----------| -| `MODAL` (100) | the open dialog's navigator | a modal dialog holds the keyboard | routes Tab/Enter/Escape to the dialog's focus ring and claims every press, so a dialog owns the keyboard exclusively while it is shown | -| `PANEL` (60) | a sub-panel the keys are meant for — the sequencer's tracker grid, the order list, the voices, the converter's list of gathered recordings, the instruments panel while an audition is open | its tab is in front, its card stands open, and the sub-panel holds what the keys act on: a cursor, a row picked out, or an open audition | handles the keys its own category names and yields the combinations it does not own so a higher-reaching shortcut still wins | -| `SHORTCUT` (40) | application shortcuts (`ShortcutManager`) | always | fires the matching shortcut while no field is being edited, or whenever the shortcut is `field_transparent` | +| `MODAL` | the open dialog's navigator | a modal dialog holds the keyboard | routes Tab/Enter/Escape to the dialog's focus ring and claims every press, so a dialog owns the keyboard exclusively while it is shown | +| `PANEL` | a sub-panel the keys are meant for | its tab is in front, its card is open, and the sub-panel holds what the keys act on: a cursor, a row picked out, or an open audition | handles the keys its own category names and yields every combination it does not own, so a higher-reaching shortcut still wins | +| `SHORTCUT` | application shortcuts (`ShortcutManager`) | always | fires the matching shortcut while no field is being edited, or whenever the shortcut is `field_transparent` | -The router offers a panel the key ahead of the shortcut scope, so a panel returns `False` on any -combination it does not own — the grid yields every `Ctrl`-modified press — which is what lets -field-transparent shortcuts such as `Ctrl+PgDn` / `Ctrl+PgUp` tab-switching reach the shortcut -scope while a grid cursor is set. +The router offers a panel the key ahead of the shortcut scope. A panel therefore returns `False` on any combination it does not own. The grid, for example, yields every `Ctrl`-modified press. That lets field-transparent shortcuts, such as the tab switch, reach the shortcut scope while a grid cursor is set. ### A panel scope answers on its own tab, from an open card -A cursor, a picked row and an open audition all outlive a move to another tab and a collapsed card, -so a panel is given the predicate that reports whether its tab is the one in front and reads it, with -its card's collapse, at the moment of the press, the way focus is read. The composition root resolves -the tab and the scope composes the answer into its `active`, which keeps the fact in one place and -leaves the router's contract — the scope decides whether it wants the key — as it stands. +A cursor, a picked row and an open audition all outlive a move to another tab and a collapsed card. A panel is therefore given a predicate that reports whether its tab is the one in front. It reads the predicate, and its card's collapse, at the moment of the press, as it reads focus. The composition root resolves the tab, and the scope composes the answer into its `active`. That keeps the fact in one place and leaves the router's contract as it stands: the scope decides whether it wants the key. -A shortcut reaching a panel's selection from the `SHORTCUT` scope asks the panel the same question: -the channel keys reach the converter's picked row only while its list would take a key itself. +A shortcut in the `SHORTCUT` scope that reaches a panel's selection asks the panel the same question. The channel keys reach the converter's picked row only while its list would take a key itself. ### Focus is pulled, not pushed -Whether a text or value field keeps a plain key for itself is one router query, `is_field_focused`, -that reads the focused item from DearPyGui at the moment of the press and counts it while that item -is actively being edited. Every input is covered by construction, and the router alone holds the -rule. +Whether a text or value field keeps a plain key for itself is one router query, `is_field_focused`. It reads the focused item from DearPyGui at the moment of the press and counts it while that item is actively being edited. Every input is covered by construction, and the router alone holds the rule. -The query resolves the focused item to the field behind it. A `dpg.group` reports the state of the -widget inside it, and DearPyGui names the outermost such group as the focused item — the -instruments panel's sequence input, laid out beside its copy button inside a card body group, -reaches the keyboard as that group. An active group therefore answers with the field being edited -below it, found by following the one branch that reports focus, so a panel-spanning group costs a -key press only the path down to its field. +The query resolves the focused item to the field behind it. A `dpg.group` reports the state of the widget inside it, and DearPyGui names the outermost such group as the focused item. The instruments panel's sequence input, laid out beside its copy button inside a card body group, reaches the keyboard as that group. An active group therefore answers with the field being edited below it. The query follows the one branch that reports focus, so a panel-spanning group costs a key press only the path down to its field. -### The modal stack +**Focus is claimed per key.** A focused input keeps the keys it genuinely consumes and yields the rest. A text or number field consumes `Space` and `Shift+Space`, because space is a character it types, and `Escape`, which cancels the field. Those keys serve the field while it holds focus. A modified combination stays global and fires from anywhere, which is why playing from the shown frame works while typing. Playing from the cursor row belongs to the grid: the sequencer grid claims it while the grid itself holds the keyboard. -The router holds a LIFO stack of modal handlers; `push_modal` / `pop_modal` bracket a dialog's -lifetime, and the built-in `MODAL` scope routes each press to the top of the stack. Since `MODAL` -outranks the panel and shortcut scopes, every scope beneath it reads the keyboard as though the -application held no dialogs at all. +**Interactive widgets release the keyboard.** A selectable cell or a transport button hands focus back after its click, so the next playback key reaches the router. `Space` and `Escape` therefore stay live in the moment after any click. -Its one global handler is bound in `shell.py` once the DPG context exists, on the router the -composition root built and injected into every consumer (architecture principle 7). +### The modal stack + +The router holds a LIFO stack of modal handlers. `push_modal` and `pop_modal` bracket a dialog's lifetime, and the built-in `MODAL` scope routes each press to the top of the stack. `MODAL` outranks the panel and shortcut scopes, so every scope beneath it reads the keyboard as though the application had no dialogs at all. --- ## The vocabulary -One key table (`utils/gui/keyboard/keys.py`) reads a key both ways — the name a file writes and the -code a press carries — and one combination type, `KeyCombination`, parses that spelling, displays -it, and answers whether a press matches it. - -Above them stands the one declared binding (architecture principle 12). The menu printing an -accelerator, the panel acting on a press, and the dispatcher firing the callback all read that one -entry, so each of the three shows or fires whatever the scheme currently says. +One key table (`utils/gui/keyboard/keys.py`) reads a key both ways: the name a file writes and the code a press carries. One combination type, `KeyCombination`, parses that spelling, displays it, and answers whether a press matches it. -**The combination is data and the category is code.** Which keys reach an action is the reader's to -choose, while which scope answers them follows from where the action is handled. A scheme is -validated as it loads — every `ShortcutId` is answered, every key name resolves, and one -combination reaches one action within a category — and a collision is a `SystemError` at startup, -beside the layout and palette failures. +**The combination is data and the category is code.** Which keys reach an action is the reader's to choose, and which scope answers them follows from where the action is handled. A scheme is validated as it loads: every `ShortcutId` is answered, every key name resolves, and one combination reaches one action within a category. A collision is a `SystemError` at startup, beside the layout and palette failures. --- @@ -101,83 +60,47 @@ beside the layout and palette failures. ### A preference layers over the shipped scheme -`ShortcutsConfig` holds the scheme name and the per-action overrides, both written the way a -keybinding file writes them, so a preference outlives the build that stored it: -`ShortcutCatalog.select` answers with the default for a scheme a build stopped shipping, and an -override naming an action this build has none of, a key the table has none of, or a combination its -category already gives away is reported and left out, so one stale entry costs only itself. +`ShortcutsConfig` holds the scheme name and the per-action overrides, both written the way a keybinding file writes them, so a preference outlives the build that stored it. `ShortcutCatalog.select` answers with the default for a scheme a build stopped shipping. An override is reported and left out when it names an action this build does not have, a key the table does not have, or a combination its category already gives away. One stale entry therefore costs only itself. -A change reaches the running application through `ShortcutSource.on_bindings_changed` — the -keyboard's analog of the palette switch ([`palette.md`](palette.md)) — and the dispatcher -re-reads the keys while the menus re-print their accelerators. Each registration names the action it -fires, which is what leaves a rebind that little to catch up. +A change reaches the running application through `ShortcutSource.on_bindings_changed`, the keyboard's analog of the palette switch ([`palette.md`](palette.md)). The dispatcher re-reads the keys, and the menus re-print their accelerators. Each registration names the action it fires, so a rebind has little to catch up. ### A scheme is edited through a draft -`ShortcutDraft` (`utils/gui/shortcuts/draft.py`) holds the scheme being edited together with the -actions the reader has touched — the combination each was given, or nothing where it was left -unbound — so what reaches the preference is those actions alone while every other key follows the -scheme beneath. +`ShortcutDraft` (`utils/gui/shortcuts/draft.py`) holds the scheme being edited together with the actions the reader has touched: the combination each was given, or nothing where it was left unbound. Only those actions reach the preference, and every other key follows the scheme beneath. -An assignment displaces: giving an action a combination its category already answers takes the key -from the holder in the same step, which is what makes every scheme a draft produces a valid one, -and the dialog names the holder and asks before that step is taken. The draft is what the dialog -edits, and a commit is what activates it, so a reader rebinding Escape, Tab or Enter keeps the keys -the dialog is operated by until they are done. +An assignment displaces. Giving an action a combination its category already answers takes the key from the holder in the same step, so every scheme a draft produces is valid. The dialog names the holder and asks before that step is taken. The dialog edits the draft, and a commit activates it, so a reader rebinding Escape, Tab or Enter keeps the keys the dialog is operated by until they are done. ### A scheme belongs to a platform; an action does not -`ShortcutId` and `ShortcutCategory` are the same on every platform, and `PLATFORM_SCHEME_NAMES` -(`constants/keybindings.py`) states which scheme each one ships — the choice a profile makes once, -at creation, after which the stored name selects. The modifier table reads every spelling on every -platform while `Modifier.SUPER` displays as the name the machine is labeled with, so a scheme -written for one keyboard loads, validates and reads on another, and the completeness validation -holds every shipped scheme to the same action set. +`ShortcutId` and `ShortcutCategory` are the same on every platform. `PLATFORM_SCHEME_NAMES` (`constants/keybindings.py`) says which scheme each platform ships. A profile makes that choice once, at creation, and the stored name selects from then on. The modifier table reads every spelling on every platform, and `Modifier.SUPER` displays as the name the machine is labeled with. A scheme written for one keyboard therefore loads, validates and reads on another, and the completeness validation holds every shipped scheme to the same action set. --- ## Actions -Declaring an action is a chain of four links, and the `shortcut-actions` check holds every one of -them (see [`architecture.md`](../architecture.md) § Enforcement): +Declaring an action takes four links, and the `shortcut-actions` check holds every one of them (see [`architecture.md`](../architecture.md#enforcement)): -| Link | Where | What it states | +| Link | Where | What it says | |------|-------|----------------| | The action | `utils/gui/shortcuts/ids.py` | its name, and the category that answers it | | Its keys | every scheme under `sampletones_config/keybindings/` | the combination that fires it, `~` where it ships unbound | -| Its call | `shell.py` — a `ShortcutBindings` field and the entry naming it in the binding map, or membership of `FAMILY_SHORTCUT_IDS` | the one call the action makes | +| Its call | `shell.py`: a `ShortcutBindings` field and the entry naming it in the binding map, or membership of `FAMILY_SHORTCUT_IDS` | the one call the action makes | | Its label | a `KeybindingActionElements` member and its `en.yaml` entry | how the keybindings editor lists it | -Two kinds of action state their call differently, and the check knows both. +Two kinds of action give their call differently, and the check knows both. -A **family** is an action a whole enum parameterizes — an export item per format, an item per -channel: a `Dict[Enum, ShortcutId]` in `ids.py` whose reader dispatches on the enum member. -`FAMILY_SHORTCUT_IDS` names the mappings that are families, so what excuses an action from stating -a call of its own is written down. `SHORTCUT_IDS_BY_NAME` answers with every action and stands -outside that list. +A **family** is an action that a whole enum parameterizes, such as an export item per format or an item per channel. It is a `Dict[Enum, ShortcutId]` in `ids.py` whose reader dispatches on the enum member. `FAMILY_SHORTCUT_IDS` names the mappings that are families, so what excuses an action from giving a call of its own is written down. `SHORTCUT_IDS_BY_NAME` answers with every action and stays outside that list. -A **panel-scope** action states no call at all, because its key scope acts on the press itself. A -`DIALOG` action is named nowhere in the editor, since a dialog is operated by the keys its category -holds. +A **panel-scope** action gives no call at all, because its key scope acts on the press itself. A `DIALOG` action is named nowhere in the editor, since a dialog is operated by the keys its category holds. ### A menu item is a view of an action -`ShortcutManager.add_menu_item(shortcut_id, ...)` is how a menu names an action: it takes both the -accelerator and the call from the action, and keeps the item under it, so a rebind re-prints the key -already on screen. An item passes a `callback` of its own only where it carries a state to show, and -then that call is the one switching the state it shows. +`ShortcutManager.add_menu_item(shortcut_id, ...)` is how a menu names an action. It takes both the accelerator and the call from the action and keeps the item under it, so a rebind re-prints the key already on screen. An item passes a `callback` of its own only where it carries a state to show, and that call is then the one that switches the state it shows. ### A set of actions several menus show is declared by whoever owns them -The owner states one builder — `GUISequencerVoicesPanel.add_action_items` for a voice, a grid's edit -surface for a cell — and each door decides where to print it: the panel's own row menu, the menu -bar's **Edit** group through `EditSurfaceProtocol` and `EditRouter`, the **Voice** group through the -panel. Adding an action to the builder reaches every door, and the dividers around it belong to the -door rather than to the set. +The owner writes one builder, such as `GUISequencerVoicesPanel.add_action_items` for a voice or a grid's edit surface for a cell. Each menu that shows the set decides where to print it: the panel's own row menu, the menu bar's **Edit** group through `EditSurfaceProtocol` and `EditRouter`, or the **Voice** group through the panel. Adding an action to the builder reaches every menu, and the dividers around it belong to the menu and not to the set. -### A menu whose contents follow a selection states them when it is opened +### A menu whose contents follow a selection is filled when it is opened -A menu bar is built once, while what an item should say follows the cursor at the moment a reader -opens the menu. `ui/elements/menu_section.py::MenuSection` is that mechanism: a marker leads the -menu, the framework reports it drawn once a frame while the menu stands open, and a gap in those -reports marks a fresh opening and restates the section. `MenuSection` states why the marker leads. +A menu bar is built once, while what an item should say follows the cursor at the moment a reader opens the menu. `MenuSection` (`ui/elements/menu_section.py`) is the mechanism. A marker leads the menu, and the framework reports it drawn once a frame while the menu is open. A gap in those reports marks a fresh opening and refills the section. The `MenuSection` docstring says why the marker leads. diff --git a/docs/development/application/palette.md b/docs/development/application/palette.md index 8ec3ed2be..144e1fcd7 100644 --- a/docs/development/application/palette.md +++ b/docs/development/application/palette.md @@ -11,30 +11,29 @@ token, resolved where it is drawn. This document holds the mechanism. ## A color is a token -Every annotation names `BaseColor` (`utils/palette/colors/`) — a dataclass field, a signature, a -dictionary key — and `WrittenColor` appears only on the Pydantic field that validates a YAML entry, -which is the one place a written token is read out of the configuration. The `rgba` read happens -where the value is handed to a widget, and what a consumer keeps is the token, so whoever holds a -color follows a palette swap. +Every annotation names `BaseColor` (`utils/palette/colors/`): a dataclass field, a signature, a dictionary +key. `WrittenColor` appears only on the Pydantic field that validates a YAML entry, the one place a written +token is read out of the configuration. The `rgba` read happens where the value is handed to a widget, and +a consumer keeps the token, so whoever holds a color follows a palette swap. ## A shade is composed by naming its form -`utils/palette/colors/` is a flat star: `base.py` declares the abstract `rgba`, and each form is a -peer module beside it (`literal`, `named`, `faded`, `grayscale`, `blended`, `layered`), answering -with a `BaseColor` of its own — `FadedColor(color=GrayscaleColor(color=token), fraction=0.3)`. Every -form is a module-level frozen dataclass, so two identical compositions are one value and a theme -cache keyed on a shade hits. +`utils/palette/colors/` has one module per form. `base.py` declares the abstract `rgba`, and each form is a +peer module beside it (`literal`, `named`, `faded`, `grayscale`, `blended`, `layered`) that answers with a +`BaseColor` of its own, for example `FadedColor(color=GrayscaleColor(color=token), fraction=0.3)`. Every +form is a module-level frozen dataclass, so two identical compositions are one value and a theme cache +keyed on a shade hits. ## A palette change is one switch -What DearPyGui has already taken a copy of is registered rather than remembered by whoever set it. -`PaletteBindings` (`utils/gui/palette/`) records each `(item, argument)` a palette color reached, and -`dpg_set_palette_color` / `dpg_add_palette_theme_color` are how a color gets there. +DearPyGui copies a color when it receives it. `PaletteBindings` (`utils/gui/palette/`) therefore records +each `(item, argument)` a palette color reached, and `dpg_set_palette_color` and +`dpg_add_palette_theme_color` are how a color gets there. -`PaletteSource.activate` then fires the composition root's listener, which re-applies the bindings, -refreshes the viewport clear color, and repaints the sequencer for the row and cell highlights -DearPyGui holds as table state. +`PaletteSource.activate` then fires the composition root's listener. The listener re-applies the bindings, +refreshes the viewport clear color, and repaints the sequencer for the row and cell highlights DearPyGui +keeps as table state. -The `palette-colors` hook holds all three rules — an attribute assigned a resolved `rgba`, a theme -color filled outside the palette bindings, and a hex literal in the shipped configuration outside -`palettes/` (see [`architecture.md`](../architecture.md) § Enforcement). +The `palette-colors` hook reports each of these: an attribute assigned a resolved `rgba`, a theme color +filled outside the palette bindings, and a hex literal in the shipped configuration outside `palettes/` +(see [`architecture.md`](../architecture.md#enforcement)). diff --git a/docs/development/application/playback.md b/docs/development/application/playback.md index 6c31e7553..c2aa7372e 100644 --- a/docs/development/application/playback.md +++ b/docs/development/application/playback.md @@ -1,277 +1,122 @@ # Playback and Transport -This document governs sound across the application: what may be heard, who decides, and what each -transport command means. Consult it when adding audio a user can start, a surface that starts it, or -a control over what is heard. The contracts here bind every tab and every player. It complements -`docs/development/architecture.md`, `docs/development/application/keyboard.md` (which owns the keyboard-routing -layer) and `docs/development/guidelines.md`. +This document governs sound across the application: what may be heard, who decides, and what each transport command means. Consult it when adding audio a user can start, a surface that starts it, or a control over what is heard. The contracts here bind every tab and every player. It complements [`architecture.md`](../architecture.md), [`keyboard.md`](keyboard.md) (which owns the keyboard-routing layer) and [`guidelines.md`](../guidelines.md). + +Two terms recur. A **source** is what plays one kind of audio: a reconstruction, an instruction or the sequencer song. The **transport** is the shared play, pause and stop control over them. --- ## Principles -1. **The output is one scarce resource, arbitrated by intent.** There is a single device, a single - stream, and a single thing sounding. Audio a user asked a tab to play outranks audio a click - auditioned, so every request states which kind it is and the arbiter — not the caller's - eagerness — decides what is heard. -2. **Ownership is the state.** Which source owns the live stream is the one fact that answers "what - is playing": commands, labels, and indicators all derive what they do and show from it, so they - agree by construction. -3. **A command addresses one target, resolved from where the user is.** Context decides which source - a verb means, and one resolution serves every verb, so the same key means the same thing on a - given screen every time. -4. **Surfaces describe the target; the transport decides.** The menu, the toolbar, and the keyboard - reach identical verbs and report identical state, so a new surface adds another way in to the - same behavior. -5. **Listening choices stay out of the document.** What the user chooses to hear is session state; - what the project holds is the whole song. Saving, export, rendering, and history read the - document, so each of them works on the full song whatever the user is listening to. A render - reads the document as it stood when it was asked for: every channel sounding, at unity gain, - played through once. -6. **Live state is pulled while sound is produced.** A player reads the settings that shape its - sound as it renders, so a change is heard as the render-ahead buffer drains. This is what lets a - listening control take effect inside the sound already playing. -7. **A row's duration belongs to the song, not to the player.** How long a row lasts follows from - the project's tempo and meter together with the row's place in the pattern, so it is a function - of position: the same row lasts the same time however playback reached it, and a module exported - from the song can state the same figures. The integer tick counts the groove places *are* the - tempo, so a render realizes them exactly at every rate it offers. +1. **The output is one scarce resource, arbitrated by intent.** There is a single device, a single stream and a single thing sounding. Audio a user asked a tab to play outranks audio a click auditioned. Every request says which kind it is, and the arbiter decides what is heard, not the caller's eagerness. +2. **Ownership is the state.** Which source owns the live stream is the one fact that says what is playing. Commands, labels and indicators all derive what they do and show from it, so they agree by construction. +3. **A command addresses one target, resolved from where the user is.** Context decides which source a verb means, and one resolution serves every verb. The same key therefore means the same thing on a given screen every time. +4. **Surfaces describe the target, and the transport decides.** The menu, the toolbar and the keyboard reach identical verbs and report identical state, so a new surface adds another way in to the same behavior. +5. **Listening choices stay out of the document.** What the user chooses to hear is session state, and what the project holds is the whole song. Saving, export, rendering and history read the document, so each of them works on the full song whatever the user is listening to. A render reads the document as it stood when it was asked for: every channel sounding, at unity gain, played through once. +6. **Live state is pulled while sound is produced.** A player reads the settings that shape its sound as it renders, so a change is heard as the render-ahead buffer drains. A listening control therefore takes effect inside the sound already playing. +7. **A row's duration belongs to the song, not to the player.** How long a row lasts follows from the project's tempo and meter together with the row's place in the pattern. It is a function of position: the same row lasts the same time however playback reached it, and a module exported from the song can state the same figures. The integer tick counts the [groove](../../glossary.md#groove) places *are* the tempo, so a render realizes them exactly at every rate it offers. ## Two kinds of sound -**Preview** — a quick audition fired by a click or a key: a file or reconstruction in a browser -tree, a voice in the sequencer, or the instrument the Reconstructions tab has open, sounded at the -note a piano key names. A preview is ephemeral. It sounds once, holds the device without claiming -ownership of it, and is meant to be heard and forgotten. It yields the device to intentional -playback, and it answers to Stop. - -An instrument's audition is a preview of that kind. It reads the generator chosen on the -instruments card, takes the note from the keyboard's two octaves above the octave the tracker -types in, and renders the voice through the same two steps a tracker row takes — the step from the -instrument's own pitch to the note, at full volume. The plot card draws the same rendering at the -pitch the instrument stands at, so what is seen and what is heard name one generator. - -**A voice sounded on its own runs to its release, or to the length it is offered.** A row holds a -voice for as long as the pattern asks, while an audition and the voice list's preview have no row -behind them, so each states a span of its own. A volume dimension ending at silence releases the -voice, and that release is where the sound stops; one circling from a loop point never reaches a -last item, so it sounds for the ticks `AUDITION_TICKS` offers it. Both spans are counted by -`audition_ticks` (`sampletones_core/performance/audition.py`), so the plot card draws exactly the -frames the keyboard sounds. - -**A sounding voice is marked where it has reached.** The device reports its position while it -plays, and the plot card carries that mark along the voice it drew, the same mark a reconstruction's -playback moves. A preview follows its own sound alone: `play` reports whether the request took the -output, and the audition starts following only when it did, so one that yields to playback the -reader asked for leaves that playback's mark where it is. The device reports a final zero as it -winds down, which takes the mark off the card. - -A report comes from the thread writing the audio and the mark is a widget, so every report crosses to -the render thread before it moves anything (architecture principle 6). A source's player reads where -its own playback stands when the report arrives, which is what lets a seek made in the meantime stand: -the mark shows the device as it is, and a report that set out before the seek draws the seek. - -**Intentional playback** — the audio a tab is built around: a reconstruction's audio, an -instruction's audio, or the sequencer song. It is owned by the source that started it, and it is -resumable, seekable, and stoppable. One intentional source at most is engaged at any moment. - -Priority ranks the two kinds and settles every contest for the device: starting intentional playback -preempts a sounding preview, and a preview requested while intentional playback holds the device is -declined. +**Preview** is a quick audition fired by a click or a key: a file or reconstruction in a browser tree, a voice in the sequencer, or the instrument the Reconstructions tab has open, sounded at the note a piano key names. A preview is ephemeral. It sounds once, holds the device without claiming ownership of it, and is meant to be heard and forgotten. It yields the device to intentional playback, and it answers to Stop. + +An instrument's audition is a preview of that kind. It reads the generator chosen on the instruments card, takes the note from the keyboard's two octaves above the octave the tracker types in, and renders the voice through the same two steps a tracker row takes: the step from the instrument's own pitch to the note, at full volume. The plot card draws the same rendering at the pitch the instrument stands at, so what is seen and what is heard name one generator. + +**A preview sounds the frames it renders.** An instrument sounds the envelopes it has, and a sample previews the frames its reconstruction recorded. Both sound exactly what they carry, because a recording's drive matters only while the conversion runs. + +**A voice sounded on its own runs to its release, or to the length it is offered.** A row holds a voice for as long as the pattern asks. An audition and the voice list's preview have no row behind them, so each has a span of its own. A volume dimension that ends at silence releases the voice, and the sound stops there. A voice that circles from a loop point never reaches a last item, so it sounds for the ticks `AUDITION_TICKS` offers it. `audition_ticks` (`sampletones_core/performance/audition.py`) counts both spans, so the plot card draws exactly the frames the keyboard sounds. + +**A sounding voice is marked where it has reached.** The device reports its position while it plays, and the plot card carries that mark along the voice it drew, as it does for a reconstruction's playback. A preview follows its own sound alone: `play` reports whether the request took the output, and the audition starts following only when it did. A request that yields to playback the reader asked for leaves that playback's mark where it is. The device reports a final zero as it winds down, which takes the mark off the card. + +A report comes from the thread writing the audio, and the mark is a widget, so every report crosses to the render thread before it moves anything (architecture principle 6). A source's player reads where its own playback stands when the report arrives. A seek made in the meantime therefore stands: the mark shows the device as it is, and a report that set out before the seek draws the seek. + +**Intentional playback** is the audio a tab is built around: a reconstruction's audio, an instruction's audio or the sequencer song. The source that started it owns it, and it is resumable, seekable and stoppable. At most one intentional source is engaged at any moment. + +Priority ranks the two kinds and settles every contest for the device. Starting intentional playback preempts a sounding preview, and a preview requested while intentional playback holds the device is declined. ## Engagement -A source is **engaged** while it owns the device output, whether it is sounding or held paused. A -source therefore reports itself engaged only while *its own* audio is the one on the device; while a -preview sounds, ownership rests outside every source and each of them reports itself idle. +A source is **engaged** while it owns the device output, whether it is sounding or held paused. A source therefore reports itself engaged only while *its own* audio is the one on the device. While a preview sounds, ownership rests outside every source and each of them reports itself idle. -Engagement is the ownership test of principle 2 in practice, and it is the single fact the transport -and the toolbar consult. +Engagement is the ownership test of principle 2 in practice, and it is the single fact the transport and the toolbar consult. ## The target -The transport acts on one **target**, resolved in two steps: the active tab's own source when that -tab has one to play — a loaded reconstruction, a loaded instruction, the song of an open project — -and otherwise whatever source is engaged. The Main tab plays only previews, so its target is always -the source engaged elsewhere, if any. - -In one line: the transport controls the tab you are on when it has something to play, and otherwise -controls what is already sounding. +The transport acts on one **target**, resolved in two steps. The active tab's own source is the target when that tab has one to play: a loaded reconstruction, a loaded instruction or the song of an open project. Otherwise the target is whatever source is engaged. The Main tab plays only previews, so its target is always the source engaged elsewhere, if any. The transport thus controls the tab you are on when it has something to play, and otherwise controls what is already sounding. -Starting a tab's idle source takes the device over, from a preview or from a source engaged -elsewhere, and Stop is how to silence background audio and leave the device idle. Play passes over a -sounding preview whenever the active tab has a source of its own, so a preview answers to Stop alone. +Starting a tab's idle source takes the device over, from a preview or from a source engaged elsewhere. Stop silences background audio and leaves the device idle. Play passes over a sounding preview whenever the active tab has a source of its own, so a preview answers to Stop alone. ## The verbs -The transport's verbs are reached identically from the Playback menu, the toolbar, and the keyboard: - -| Key | Command | Behavior | -|-----|---------|-----------| -| `Space` | Play / Pause | Acts on the target: pauses or resumes it while it is engaged, and starts it from the beginning otherwise. With no target, it does nothing. | -| `Shift+Space` | Play from start | Starts the active tab's source from the beginning. | -| `Ctrl+Space` | Play from this frame | Sequencer: plays the song from the first row of the frame the tracker shows. | -| `Ctrl+Shift+Space` | Play from here | Sequencer panels: plays the song from the cursor's row. | -| `Escape` | Stop | Silences everything — the engaged source and any preview — from any tab. | - -**A click on a waveform puts the playhead at a sample.** The Reconstructions and Instructions -waveforms draw the audio their tab's own source plays, so a click reaches that source at the sample -under the pointer: an engaged source moves there and goes on sounding or stays paused, and an idle -one starts sounding there. The seek and the ownership it depends on are read under one lock, so a -source another has taken the output from starts sounding rather than moving a playback it lost. A -waveform drawing a voice's own audio draws nothing its player sounds, so it takes no click and its -hint names none. - -The left button carries three gestures. A drag pans the view, so a press reads as a click while the -pointer comes up within the waveform layout's `click_travel` of where it went down. A double-click -fits the view to the audio (the plot's `fit_button`), so a click is reported once its double-click -window has closed. The double-click is the one ImGui recognizes, read on the press that completes it, -and every press following it within the window belongs to the same burst, so the playhead and the -view each answer exactly the gesture meant for them (`PlotClickGesture`, -`ui/elements/graphs/gesture.py`). - -Because the target prefers the active tab's own source, `Space` controls what the user is looking at -whenever that screen can play something, and reaches the source already sounding on a screen that -plays nothing of its own: the Main tab, an empty Reconstruction or Instructions tab, the Sequencer -before a project is open. So a paused reconstruction resumes with `Space` from the Main tab, while -`Space` on the Sequencer with a project open starts the song. +The transport's verbs are reached identically from the Playback menu, the toolbar and the keyboard: + +| Command | Behavior | +|---------|-----------| +| Play / Pause | Acts on the target: pauses or resumes it while it is engaged, and starts it from the beginning otherwise. With no target, it does nothing. | +| Play from start | Starts the active tab's source from the beginning. | +| Play from this frame | Sequencer: plays the song from the first row of the frame the tracker shows. | +| Play from here | Sequencer panels: plays the song from the cursor's row. | +| Stop | Silences everything, the engaged source and any preview, from any tab. | + +Each verb is an action, so the combination it answers to is the shipped scheme's (`sampletones_config/keybindings/`) and every surface prints what the scheme says. + +**A click on a waveform puts the playhead at a sample.** The Reconstructions and Instructions waveforms draw the audio their tab's own source plays, so a click reaches that source at the sample under the pointer. An engaged source moves there and goes on sounding or stays paused, and an idle one starts sounding there. The seek and the ownership it depends on are read under one lock, so a source another has taken the output from starts sounding and does not move a playback it lost. A waveform that draws a voice's own audio draws nothing its player sounds, so it takes no click and its hint names none. + +The left button carries three gestures: a click seeks, a drag pans the view, and a double-click fits it to the audio. `PlotClickGesture` (`ui/elements/graphs/gesture.py`) settles which one a press was, so the playhead and the view each answer only the gesture meant for them. + +Because the target prefers the active tab's own source, Play/Pause acts on what the user is looking at whenever that screen can play something. On a screen that plays nothing of its own, it reaches the source already sounding. Those screens are the Main tab, an empty Reconstruction or Instructions tab, and the Sequencer before a project is open. So it resumes a paused reconstruction from the Main tab, and starts the song on the Sequencer with a project open. ## What the surfaces show -The toolbar's transport strip and the Playback menu describe the target. The Play/Pause/Resume label -and the paused indicator report what the toggle will do; Stop is available while any sound is on the -device, engaged or previewed. So the display carries across tabs that play nothing of their own, -showing the source sounding elsewhere, and shows the local source on tabs that have one. A verb -tied to one screen — playing from the shown frame — is offered on that screen with its document -open. - -The sequencer view reports the playhead too, at the reach the **follow mode** chooses: the sounding -row, the frame that holds it, or the view the user placed. The mode is one setting with two derived -answers — whether the tracker shows the frame being played, and whether it scrolls to keep the -sounding row in sight — and those two are its whole contract, which every surface that follows the -playhead reads. The song player holds the mode and emits it with every position, which is what lets -the menu's check and the grid's scrolling settle in one step when the mode changes mid-playback. - -A mark belongs to what it names. The playhead's position is a frame and a row within it, so the -order grid marks the frame under every mode, while the row's mark reads as the sounding row of the -pattern on screen: the tracker carries it while the frame it shows is the frame that sounds, and the -mark travels with the frame across a structural order edit. Every mode paints on this rule, and the -mode governs where the view sits. - -## Keyboard delivery under field focus - -Playback keys arrive through the application's single key handler (architecture §12). These rules -keep them predictable: - -**Focus is claimed per key.** A focused input keeps the keys it genuinely consumes and yields the -rest. A text or number field consumes `Space` and `Shift+Space` (space is a character it types) and -`Escape` (which cancels the field), so those keys serve the field while it holds focus. A modified -combination stays global and fires from anywhere, which is why playing from the shown frame works -while typing. Playing from the cursor row belongs to the grid: the sequencer grid claims it while the -grid itself holds the keyboard. - -**Interactive widgets release the keyboard.** A selectable cell or a transport button hands focus -back after its click, so the next playback key reaches the router. This keeps `Space` and `Escape` -live in the moment after any click. +The toolbar's transport strip and the Playback menu describe the target. The Play/Pause/Resume label and the paused indicator report what the toggle will do. Stop is available while any sound is on the device, engaged or previewed. The display therefore carries across tabs that play nothing of their own, showing the source sounding elsewhere, and shows the local source on tabs that have one. A verb tied to one screen, such as playing from the shown frame, is offered on that screen with its document open. + +The sequencer view reports the playhead too, at the reach the **follow mode** chooses: the sounding row, the frame that holds it, or the view the user placed. The mode is one setting with two derived answers: whether the tracker shows the frame being played, and whether it scrolls to keep the sounding row in sight. Those two are its whole contract, and every surface that follows the playhead reads them. The song player holds the mode and emits it with every position, so the menu's check and the grid's scrolling settle in one step when the mode changes mid-playback. + +A mark belongs to what it names. The playhead's position is a frame and a row within it. The order grid marks the frame under every mode. The row's mark reads as the sounding row of the pattern on screen: the tracker carries it while the frame it shows is the frame that sounds, and the mark travels with the frame across a structural order edit. Every mode paints on this rule, and the mode governs where the view sits. + +## Keyboard delivery + +Playback keys arrive through the application's single key handler (architecture principle 12). [`keyboard.md`](keyboard.md) describes how a focused field claims some of them and yields the rest. ## Silencing channels -The sequencer's tracker channels can be silenced for listening. One mute set holds the silenced -channels for the open document, and everything else is derived from it: the **active-channel mask** -the song mixes through, and solo — soloing silences the other channels and remembers the mix it -replaced, so soloing the same channel again returns to that mix. Deriving both from one set is what -keeps the gestures consistent: any way of reaching "silence the rest" leaves the same state as any -other. - -Every surface shows that one set and switches it. In the tracker a channel recedes down its column; -in the order table it recedes along its row; both take their shades from one pair of colors, so a -silenced channel looks the same wherever it appears. A channel's name is the switch in both tables — -click to silence, modified click to solo, the master name for the whole mix — and both tables hand -the gesture and its right-click menu to one object, so both offer the same wording and the same -behavior. The Playback menu's **Channels** submenu carries the same set as a check per channel, plus -one item that returns the whole mix. Each of those items is registered as an action whether or not a -key is bound to it, so the keybinding scheme can give it one and the menu prints what the scheme -says (architecture principle 12). - -The mask is pulled per rendered row, which is principle 6 for this control: a channel drops in or -out as the render-ahead buffer drains, with the immediacy every other live edit has. A silenced -channel still takes each row's voice, transpose, and volume, so returning it to the mix resumes -on the state its pattern has reached. - -Muting is monitoring, and principle 5 governs what follows. The project holds every channel, so -saving, module export, and any rendered output write the full song. The history stack holds project -state alone, so undo, redo, and history jumps carry the mute set across untouched — which is why the -sequencer distinguishes a history restore from a document transition. And the mute set belongs to the -listening session, so opening, creating, or closing a document starts a fresh one with every channel -audible. +The sequencer's tracker channels can be silenced for listening. One mute set holds the silenced channels for the open document, and everything else derives from it: the **active-channel mask** the song mixes through, and solo. Soloing silences the other channels and remembers the mix it replaced, so soloing the same channel again returns to that mix. Deriving both from one set keeps the gestures consistent, because any way of reaching "silence the rest" leaves the same state as any other. + +Every surface shows that one set and switches it. A channel's name is the switch in both the tracker and the order table: click to silence, modified click to solo, the master name for the whole mix. Both tables hand the gesture and its right-click menu to one object, so they offer the same wording and the same behavior, and both take their shades from one pair of colors. The Playback menu's **Channels** submenu carries the same set as a check per channel, plus one item that returns the whole mix. Each of those items is registered as an action whether or not a key is bound to it, so the keybinding scheme can give it one and the menu prints what the scheme says (architecture principle 12). + +The mask is pulled per rendered row, which is principle 6 for this control: a channel drops in or out as the render-ahead buffer drains, with the immediacy every other live edit has. A silenced channel still takes each row's voice, transpose and volume, so returning it to the mix resumes on the state its pattern has reached. + +Muting is monitoring, and principle 5 governs what follows. The project holds every channel, so saving, module export and any rendered output write the full song. The history stack holds project state alone, so undo, redo and history jumps carry the mute set across untouched. That is why the sequencer distinguishes a history restore from a document transition. The mute set belongs to the listening session, so opening, creating or closing a document starts a fresh one with every channel audible. ## What the channel holds -A sample states every dimension of every frame, and its reconstruction names which of those -dimensions the instrument itself wrote. The rest are the channel's: each channel carries a value per -dimension — volume, arpeggio, timbre — and an instrument leaving one empty sounds it at the value the -channel holds. That is what clearing an envelope in the instruments panel means once the sample is -played in a song, and it is the same rule a FamiTracker instrument follows with a sequence left out. +A sample has a value for every dimension of every frame, and its reconstruction names the dimensions the channel governs. The instrument writes the rest itself. Each channel carries a value per dimension (volume, arpeggio, timbre), and an instrument that leaves one empty sounds it at the value the channel holds. That is what clearing an envelope in the instruments panel means once the sample is played in a song. A FamiTracker instrument follows the same rule with a sequence left out. -The value moves as the song plays. Every frame an instrument writes hands its value to the channel, -so the channel keeps the last one written and an instrument that leaves the dimension empty picks it -up. A silent frame states its level alone, leaving pitch and timbre where the channel holds them. +The value moves as the song plays. Every frame an instrument writes hands its value to the channel, so the channel keeps the last one written, and an instrument that leaves the dimension empty picks it up. A silent frame sets its level alone and leaves pitch and timbre where the channel holds them. -A pass through the song begins on the values a channel holds from the start — full volume, no -arpeggio offset, the first timbre — so starting the song and looping back to its first row both -sound the same. Seeking within a running song keeps the values, since the channel has reached them. +A pass through the song begins on the values a channel holds from the start: full volume, no arpeggio offset, the first timbre. Starting the song and looping back to its first row therefore sound the same. Seeking within a running song keeps the values, since the channel has reached them. ## Rendering the song to a file -A render writes the whole song to an audio file through the kernel that plays it. `RowSynthesizer` -serves both: the player drives it to feed the device, the render drives it to feed a file writer. -The synthesis is therefore written once, and the file and the playback agree on what the song -sounds like by construction. - -Two things differ between them, and each is stated by whoever asks for the audio. The **document** -is a seam: a kernel reads its project through `ProjectSource`, which the live controller satisfies -for playback and a frozen `ProjectSnapshot` satisfies for a render — so the player follows every -edit as the buffer drains (principle 6), while a render describes one state of the document however -the project moves on. The **rate** is the consumer's: the device for playback, the chosen output -format for a render. The kernel rebuilds its generators and its tick clock when either moves, so a -file is written at the rate its engine ran at. - -A rate is therefore asked for once there is audio to take it, which is the first row a kernel -renders: a device has been chosen by the time playback starts, and a format by the time a render -does. A session on a machine offering no output device opens on that rule, and everything that -writes rather than sounds — editing, exporting a module, rendering to a file — works on it. - -The song's exact length follows from the timing model before a sample is rendered: the order's -length in rows gives the ticks, the tick clock gives the samples those ticks span. That figure is -what the progress bar counts against and what a finished file measures. - -Rendering is an exclusive operation (architecture principle 10). It occupies the application from -the moment its dialog opens until that dialog closes, and it joins the same busy authority as -conversion and library generation, so each of the three holds the others off and every surface -offering one reads a single answer. - -The write itself takes one pass, or two where the user asks for a normalized peak: the first pass -spills raw samples and discovers the peak, the second reads them back and encodes at the scale that -peak sets. Each pass names itself, so the bar crosses one axis — samples — twice, holding a single -unit across both. A cancel is honored between rows and between encoded blocks, and a render that -is stopped or fails clears the destination and the spill, so a result names a path where a finished -file stands. +A render writes the whole song to an audio file through the synthesizer that plays it. `RowSynthesizer` serves both: the player drives it to feed the device, and the render drives it to feed a file writer. The synthesis is therefore written once, and the file and the playback agree on what the song sounds like by construction. + +Two things differ between them, and whoever asks for the audio sets each. The **document** is read through an interface. The synthesizer reads its project through `ProjectSource`, which the live controller satisfies for playback and a frozen `ProjectSnapshot` satisfies for a render. The player therefore follows every edit as the buffer drains (principle 6), while a render describes one state of the document however the project moves on. The **rate** is the consumer's: the device for playback, the chosen output format for a render. The synthesizer rebuilds its generators and its tick clock when either changes, so a file is written at the rate its engine ran at. + +A rate is asked for once there is audio to take it, which is the first row the synthesizer renders. A device has been chosen by the time playback starts, and a format by the time a render does. A session on a machine with no output device opens on that rule, and everything that writes and does not sound works on it: editing, exporting a module and rendering to a file. + +The song's exact length follows from the timing model before a sample is rendered. The order's length in rows gives the ticks, and the tick clock gives the samples those ticks span. That figure is what the progress bar counts against and what a finished file measures. + +Rendering is an exclusive operation (architecture principle 10). It occupies the application from the moment its dialog opens until that dialog closes. It joins the same busy authority as conversion and library generation, so each of the three holds the others off and every surface offering one reads a single answer. + +The write takes one pass, or two where the user asks for a normalized peak. The first pass spills raw samples and discovers the peak, and the second reads them back and encodes at the scale that peak sets. Each pass names itself, so the bar crosses one axis, samples, twice, and holds a single unit across both. A cancel is honored between rows and between encoded blocks. A render that is stopped or fails clears the destination and the spill, so a path in a result always names a finished file. ## Teardown -The device is torn down once every source holding a stream has released it. A source that streams to -the device writes from a thread of its own, so that source alone can bring the writing to a stop and -hand the stream back — and the hand-back is what leaves the backend safe to terminate. +The device is torn down once every source holding a stream has released it. A source that streams to the device writes from a thread of its own, so only that source can bring the writing to a stop and hand the stream back. The hand-back is what leaves the backend safe to terminate. -`PlaybackRouter.shutdown()` is the seam the application calls as it quits. It reaches every registered -source rather than the engaged one alone, so a source holding a stream is wound down whatever the -transport reports at that moment. +`PlaybackRouter.shutdown()` is the entry point the application calls as it quits. It reaches every registered source and not only the engaged one, so a source holding a stream is wound down whatever the transport reports at that moment. -The device holds a release per stream it handed out and invokes it whenever it needs the output free: -as the backend is torn down, and on a device change, where the release stops the song so the new -device opens cleanly. A stream that outlives its release leaves the running backend in place — the -manager reports the failure and keeps the instance, since the source still writes to memory that -terminating would reclaim. +The device holds a release per stream it handed out and invokes it whenever it needs the output free: as the backend is torn down, and on a device change, where the release stops the song so the new device opens cleanly. A stream that outlives its release leaves the running backend in place. The manager reports the failure and keeps the instance, since the source still writes to memory that terminating would reclaim. ## Who governs what @@ -280,29 +125,12 @@ terminating would reclaim. | The device, its stream, and arbitration between requests | `AudioDeviceManager` (`sampletones_core/audio/`) | | The ranking that settles a contest for the device | `PlaybackPriority` (`logic/shared/`) | | The verbs, target resolution, and the registry of sources | `coordinators/playback/router.py` | -| Putting a sample source's playhead at a clicked sample | `PlayerLogic.play_from` (`logic/shared/player.py`) | -| Winding every source down ahead of backend teardown | `PlaybackRouter.shutdown()` (`coordinators/playback/router.py`) | | A source's engagement reporting | the transport's player protocol, implemented per source | | Error presentation for a source's failures | `GuardedPlayer` (`coordinators/playback/guard.py`) | -| Keyboard delivery, priority, and field focus | `utils/gui/keyboard/` (architecture §12) | | The sequencer's mute set, its mask, and solo | `SequencerChannelsLogic` (`logic/sequencer/channels.py`) | -| A channel name's gestures and menu, in either table | `ChannelSwitch` (`ui/panels/sequencer/channels.py`) | -| The reach the sequencer view follows the playhead at | `FollowMode` (`constants/playback.py`), held by `SongPlayerLogic` (`logic/sequencer/playback/song_player.py`) | -| Where the playhead stands, and both grids' marks for it | `SequencerTabCoordinator` (`coordinators/tabs/sequencer.py`) | -| Marking and revealing the sounding row in the tracker | `GUISequencerTrackerPanel` (`ui/panels/sequencer/tracker.py`) | | Row mixing, and the mask it pulls while rendering | `RowSynthesizer` (`logic/sequencer/playback/synthesizer/`) | -| Filling in the dimensions a channel governs, frame by frame | `SampleVoice` (`logic/sequencer/playback/synthesizer/voice.py`) | | The values a channel holds between frames | `ChannelState` (`logic/sequencer/playback/synthesizer/state.py`) | -| The channel generators and the rates they are built at | `ChannelBank` (`logic/sequencer/playback/synthesizer/bank.py`) | -| How long each row of a pattern lasts | `Groove` (`sampletones_core/timing/`), indexed by row while rendering | -| How many samples one of that row's ticks spans | `TickClock` (`sampletones_core/timing/`), followed by `EngineRates` | -| The song's render-ahead buffer | `services/song_player/` | -| The document a kernel reads, live or captured | `ProjectSource` / `ProjectSnapshot` (`logic/shared/project_source.py`) | -| The ticks the order lasts and the samples they span | `SongLength` (`logic/sequencer/playback/synthesizer/length.py`) | +| How long a row lasts, and how many samples its ticks span | `Groove` and `TickClock` (`sampletones_core/timing/`) | | Rendering the song to a file, its passes and its progress | `SongRenderService` (`services/render/`) | -| Where a rendered file's samples go, normalized or direct | `RenderSink` (`services/render/sink.py`) | -| The choices a render is made under, and the phase it is in | `SongRenderLogic` (`logic/render/`) | -| The formats a file may be written in, and what each accepts | `sampletones_core/audio/writers/` | -The sequencer song is an ordinary intentional source alongside the reconstruction and instruction -players: it implements the same protocol and is arbitrated by the same rules. +The sequencer song is an ordinary intentional source alongside the reconstruction and instruction players. It implements the same protocol and is arbitrated by the same rules. diff --git a/docs/development/application/render-thread.md b/docs/development/application/render-thread.md index 4cff27662..c8e3896b7 100644 --- a/docs/development/application/render-thread.md +++ b/docs/development/application/render-thread.md @@ -13,67 +13,68 @@ context belongs to the render thread. This document holds the mechanism. ## A background result crosses through `CallbackQueue` -Services execute long-running work on background threads and post each result to `CallbackQueue` -with a priority; the main-thread render loop drains the due results each frame within a per-frame -time budget (`scheduling.queue_budget_seconds`), so a large backlog spreads across frames while -rendering continues. Every background result reaches UI state this way, and applying one to UI state -directly from the worker thread is forbidden. A logic object hearing a worker's report — a library -generation, the audio device's position — posts its own handler to the queue the same way, since the -logic layer reaches the queue and leaves `utils/gui` to the visual layers. +Services run long work on background threads and post each result to `CallbackQueue` with a priority. The +main-thread render loop **drains** the queue: each frame it runs the due results within a per-frame time +budget (`scheduling.queue_budget_seconds`), so a large backlog spreads across frames while rendering +continues. Every background result reaches UI state this way. Applying one to UI state directly from the +worker thread is forbidden. + +A logic object hearing a worker's report, such as a library generation or the audio device's position, +posts its own handler to the queue the same way. The logic layer reaches the queue, and `utils/gui` +belongs to the visual layers. ## Work arriving from a worker crosses through `on_render_thread` -A thread of our own — a directory being read, a subtree being rebuilt — reaches the interface while -the render thread is walking the very items it would create and drop, and an item freed there is -freed with no Python thread state: a crash rather than a glitch. -`utils/gui/render_thread.py::on_render_thread` is that crossing: work already on the render thread -runs where it stands, and work arriving from any other thread joins the queue. A worker that reads a -value or sets one on a standing widget still goes through it, since the hazard is the thread rather -than the gesture. +A thread of our own, such as a directory being read or a subtree being rebuilt, reaches the interface while +the render thread is walking the very items it would create and drop. An item freed there is freed with no +Python thread state, which crashes the process. + +`on_render_thread` (`utils/gui/render_thread.py`) is the crossing. Work already on the render thread runs +where it stands, and work arriving from any other thread joins the queue. A worker that reads a value or +sets one on a standing widget still goes through it, since the hazard is the thread and not the gesture. -A run claims the drawing thread when its loop starts and lets it go when the loop stops. An -unclaimed context runs the work in place, which is what an interface being built stands in. +A run claims the drawing thread when its loop starts and lets it go when the loop stops. Where no run has +claimed the thread, as while the interface is being built, the work runs in place. ## A widget's own gesture is held for the frame -DearPyGui answers a gesture on a thread of its own, so a callback that rebuilds widgets there runs -while the render loop walks the very items it drops. `utils/gui/callbacks.py::hold_callbacks` turns -on manual callback management when the context is created, and `run_held_callbacks` runs what -DearPyGui gathered at the top of each frame's drain. So a gesture reaches the interface from the -thread that drew it, and `on_render_thread` is a direct call inside a callback because the callback -already stands there. +DearPyGui answers a gesture on a thread of its own, so a callback that rebuilds widgets there runs while +the render loop walks the very items it drops. `hold_callbacks` (`utils/gui/callbacks.py`) turns on manual +callback management when the context is created, and `run_held_callbacks` runs what DearPyGui gathered at +the top of each frame's drain. A gesture therefore reaches the interface from the thread that drew it, and +`on_render_thread` is a direct call inside a callback, because the callback already runs there. + +What a gesture costs is paid between frames. A callback heavy enough to be felt should spread its work +across frames itself. ## A gesture that waits keeps the frames going -A callback standing on the render thread holds the frames up for as long as it runs, and a native -dialog runs for as long as the reader takes to answer it. -`utils/gui/render_thread.py::answered_while_drawing` puts that waiting on a thread of its own and -draws frames until it reports back, which is how `utils/file_dialogs/api.py` opens one. The gestures -those frames gather wait for the drain that follows, so the interface stays painted while it stands -inert. +A callback on the render thread holds the frames up for as long as it runs, and a native dialog runs for as +long as the reader takes to answer it. `answered_while_drawing` (`utils/gui/render_thread.py`) puts that +waiting on a thread of its own and draws frames until it reports back. `utils/file_dialogs/api.py` opens +native dialogs this way. The gestures those frames gather wait for the drain that follows, so the interface +stays painted while it is inert. ## Work that needs a drawn frame names the frame it waits for -Reading a laid-out size or letting a configuration take effect needs a frame to have been drawn with -it, while the drain runs between frames rather than inside one. -`FrameCallbackManager.set_frame_callback` (`utils/gui/frame.py`) names the frame the work is picked -up on, and is how a callback waits for one. +Reading a laid-out size, or letting a configuration take effect, needs a frame to have been drawn with it. +The drain runs between frames and not inside one. `FrameCallbackManager.set_frame_callback` +(`utils/gui/frame.py`) names the frame the work is picked up on, and a callback uses it to wait for one. -Work that needs a particular item drawn waits on that item instead: an item visible handler reports -each frame DearPyGui draws the item in and no frame it stands hidden in — a collapsed card, a tab in -the back — so standing the handler on and off makes it the clock of work that follows the item on -screen. The stems list settles its windowed region this way. +Work that needs a particular item drawn waits on that item instead. An item visible handler reports each +frame DearPyGui draws the item in and no frame it is hidden in, such as a collapsed card or a tab in the +back. Standing the handler on and off makes it the clock of work that follows the item on screen. The stems +list settles its windowed region this way. -The drain is what makes the wait a scheduled one. The render thread inside a drain is between frames -rather than inside one, which makes the next frame the drain's own to reach, so `dpg.split_frame` -there waits for what the wait itself prevents and the application stops for good. Naming a frame -count asks for the same thing and lets the loop keep running. +The drain makes the wait a scheduled one. The render thread inside a drain is between frames, so the next +frame is the drain's own to reach. `dpg.split_frame` there waits for the very frame the wait itself +prevents, and the application stops for good. Naming a frame count with `set_frame_callback` asks for the +same wait and lets the loop keep running. ## Delayed work goes through the queue -To change the interface after a delay, post the change with `utils/callbacks/delay.py::call_after`. -The change waits in the queue until the delay has passed, and then runs on the render thread. +To change the interface after a delay, post the change with `call_after` (`utils/callbacks/delay.py`). The +change waits in the queue until the delay has passed, and then runs on the render thread. -Work still waiting in the queue at shutdown is discarded. This prevents delayed work from accessing -the interface after the context has been closed. A separate timer thread could otherwise make such an -access and cause a crash. +Work still waiting in the queue at shutdown is discarded, so delayed work never reaches the interface after +the context has closed. A separate timer thread could make such an access and crash. diff --git a/docs/development/application/sequencer-blocks.md b/docs/development/application/sequencer-blocks.md index 4784c7819..77d4f4e29 100644 --- a/docs/development/application/sequencer-blocks.md +++ b/docs/development/application/sequencer-blocks.md @@ -1,14 +1,8 @@ # Sequencer blocks -A **block** is a rectangle of one sequencer grid, lifted out of the song so it can be -written back somewhere else. Copy, cut, paste and delete are the four gestures over it, -and both grids — the tracker's pattern rows and the order's frames — carry the same set. +A **block** is a rectangle of one sequencer grid, lifted out of the song so it can be written back somewhere else. Copy, cut, paste and delete are the four gestures over it, and both grids, the tracker's pattern rows and the order's frames, carry the same set. Each grid's first column is an **aggregate** that summarizes the channel columns beside it: the tracker's **Voice** column and the order's **Master** row. -This document states the rules those gestures follow, how a block leaves the app as text, -how a selection is drawn, and how a grid's actions reach the menus and the keyboard that -fire them. The layering they sit in is -[Architecture](../architecture.md); the conventions the code is held to are the -[coding guidelines](../guidelines.md). +This document describes the rules those gestures follow, how a block leaves the app as text, how a selection is drawn, and how a grid's actions reach the menus and the keyboard that fire them. Consult it when changing copy, cut, paste, delete, shifting or selecting in either grid. [Architecture](../architecture.md) describes the layering they sit in, and the [coding guidelines](../guidelines.md) hold the conventions the code follows. ## Three vocabularies, kept apart @@ -17,95 +11,61 @@ A gesture crosses three representations, and each has one owner: | Term | Where it lives | What it names | |------|----------------|---------------| | **Cursor** | `ui/panels/sequencer/input/` | Where the reader is typing, plus the anchor a selection was started from | -| **Region** / **Cell** | `view_model/sequencer/region.py` | The rectangle a gesture acts on, and the single cell a paste is anchored at — grid coordinates, inclusive bounds | +| **Region** / **Cell** | `view_model/sequencer/region.py` | The rectangle a gesture acts on, and the single cell a paste is anchored at, in grid coordinates with inclusive bounds | | **Block** | `logic/sequencer/tracker/`, `logic/sequencer/order/` | The values themselves, keyed by offsets from the cell they were read at | -A region names *where*; a block carries *what*. A block holds offsets rather than -coordinates, which is what lets it land anywhere it is anchored. +A region names *where*, and a block carries *what*. A block holds offsets and not coordinates, which lets it land anywhere it is anchored. Two axes underpin both grids: -- **`constants/sequencer.py::CHANNEL_AXIS`** — `(None,) + ChannelName.items()`. Index 0 - is the aggregate column (the tracker's **Voice**, the order's **Master**) and 1 to 4 - are the channels. Both grids lay out along it, so a row index means the same thing in - either. -- **`view_model/sequencer/slot.py::TrackerSlot`** — a column paired with a subcolumn, - readable as a single flat index. Navigation and selection walk the flat index; an edit - addresses the pair. +- **`CHANNEL_AXIS`** (`constants/sequencer.py`) lists the aggregate first and then the four channels. Index 0 is the aggregate column and 1 to 4 are the channels. Both grids lay out along it, so a row index means the same thing in either. +- **`TrackerSlot`** (`view_model/sequencer/slot.py`) pairs a column with a subcolumn and can be read as a single flat index. Navigation and selection walk the flat index, and an edit addresses the pair. ## A cell reaches a block in one of three states -The state is carried by the block's map alone, so every consumer reads it the same way: +The block's map alone carries the state, so every consumer reads it the same way: | State | In the map | Written as | |-------|-----------|------------| | A value | Key present, holding it | That value | -| Empty | Key present, holding `None` | Emptiness — the target is cleared | -| Mixed | Key absent | Nothing — the target keeps what it had | +| Empty | Key present, holding `None` | Emptiness: the target is cleared | +| Mixed | Key absent | Nothing: the target keeps what it had | -Mixed is what an aggregate cell reads when the channels beneath it disagree, the same -`?` the grid displays. Display and clipboard route through one rule, -`sampletones_shared/utils/agreement.py::Agreement`, so a block states about a cell -exactly what the table it was read from shows there. +Mixed is what an aggregate cell reads when the channels beneath it disagree, the same `?` the grid displays. Display and clipboard route through one rule, `Agreement` (`sampletones_shared/utils/agreement.py`), so a block says about a cell exactly what the table it was read from shows there. -Absence is also what settles the order's growth (below): a column a block says nothing -about reaches nothing. +Absence also settles the order's growth (below): a column a block says nothing about reaches nothing. ## Kind alignment is arithmetic -A tracker block carries subcolumn offsets measured from `column_slot_base(column)`, and -every base is a multiple of the subcolumn count. An offset therefore addresses the same -kind of subcolumn at whichever column it is replayed against: a voice reference reaches only -another voice slot. The paste hook takes a `TrackerCell` — a row and a column, with no -subcolumn — so the type states the rule: the anchor decides *where* a block lands and the -block decides *which kind* goes where. +A tracker block carries subcolumn offsets measured from `column_slot_base(column)`, and every base is a multiple of the subcolumn count. An offset therefore addresses the same kind of subcolumn at whichever column it is replayed against: a voice reference reaches only another voice slot. The paste hook takes a `TrackerCell`, a row and a column with no subcolumn, so the type carries the rule. The anchor decides *where* a block lands, and the block decides *which kind* goes where. ## A paste is a run of the single-cell edits -The writers resolve every cell to a method the grid already has: -`SequencerTrackerLogic.place_note` / `cut_note` / `set_cell_subcolumn` / -`clear_cell_subcolumn`, and `SequencerOrderLogic.write_entry`. Nothing about the aggregate -column's fan-out is restated in a writer, so a pasted cell means exactly what the same -value typed by hand means. That is why each write is explainable, and why the aggregate's -rules have one home. +The writers resolve every cell to the single-cell edit the grid already has. No writer restates the aggregate column's fan-out, so a pasted cell means exactly what the same value typed by hand means. Each write is therefore explainable, and the aggregate's rules have one home. -Two consequences follow from the order the writes are taken in: +The order the writes are taken in has two consequences: -- Within a position, the aggregate row is written before the channels beneath it, so a - channel cell in the same block overwrites what the aggregate settled. The more specific - write wins. -- In the tracker, notes land before the transposes and volumes sharing their row, because - placing a sample through the **Voice** column clears the channels of that row. +- Within a position, the aggregate row is written before the channels beneath it, so a channel cell in the same block overwrites what the aggregate settled. The more specific write wins. +- In the tracker, notes land before the transposes and volumes sharing their row, because placing a sample through the **Voice** column clears the channels of that row. ## The order grows to what a paste reaches -A block pasted past the last frame appends frames, and the rule is stated in terms of -writes rather than the block's shape: the order grows to the last position a write -actually lands at. A `?`-only overrun column appends nothing; one holding an empty cell -appends the frame it silences. Rows clipped at **Noise** take their columns' growth with -them. +A block pasted past the last frame appends frames. The rule is stated in terms of writes and not of the block's shape: the order grows to the last position a write actually lands at. A `?`-only overrun column appends nothing, and one holding an empty cell appends the frame it silences. Rows clipped at **Noise** take their columns' growth with them. -Growth runs before the first write, so one history entry covers the appended frames and -the values in them, and a single undo takes both back. Delete keeps the order's length: -emptied trailing frames stand as silent ones. +Growth runs before the first write, so one history entry covers the appended frames and the values in them, and a single undo takes both back. Delete keeps the order's length: emptied trailing frames stay as silent ones. ## A shift reads the columns behind a region -Transpose and volume move whole cells, while a region names its edges as subcolumns. A shift -therefore reads the columns a region covers (`TrackerRegion.columns`) and reaches each of their -channels once, at every row the region spans. Two consequences follow: a nudge raised with the -cursor on a volume subcolumn still moves that cell's transpose, and a region covering the sample -column together with a channel beneath it moves that channel a single step, since the sample column -stands for the channels a value typed in it writes to. +Transpose and volume move whole cells, while a region names its edges as subcolumns. A shift therefore reads the columns a region covers (`TrackerRegion.columns`) and reaches each of their channels once, at every row the region spans. Two consequences follow: -Each cell reaches the grid through the single-cell adjustment that already governs it, the way a -pasted cell does, so a shift lands exactly the writes the same nudge repeated by hand would make — -the transpose and volume ranges included. +- A nudge raised with the cursor on a volume subcolumn still moves that cell's transpose. +- A region covering the sample column together with a channel beneath it moves that channel a single step, since the sample column represents the channels a value typed in it writes to. + +Each cell reaches the grid through the single-cell adjustment that already governs it, as a pasted cell does. A shift therefore lands exactly the writes the same nudge repeated by hand would make, the transpose and volume ranges included. ## A block states itself as text -A copy also writes the block to the desktop's clipboard, as the lines the grid prints — a -tracker block: +A copy also writes the block to the desktop's clipboard, as the lines the grid prints. A tracker block: ``` SampleToNES/1 tracker rows=2 slots=3..5 @@ -120,162 +80,69 @@ SampleToNES/1 order rows=1 positions=0..1 00 03 ``` -The form and its reading live in `logic/sequencer/clipboard/`, which deals in blocks and -strings alone; the desktop's clipboard is reached through -`utils/gui/clipboard/protocol.py::TextClipboard`, one more piece of external behavior standing -behind a protocol ([Architecture](../architecture.md), principle 11). The sequencer coordinator -wires the two. +The form and its reading live in `logic/sequencer/clipboard/`, which deals in blocks and strings alone. The desktop's clipboard is reached through `TextClipboard` (`utils/gui/clipboard/protocol.py`), one more piece of external behavior behind a protocol ([Architecture](../architecture.md), principle 11). The sequencer coordinator wires the two. -**A field prints what the grid prints in its cell**, which is what carries the three states -across: a value reads as its value, an empty cell as the dots beneath it, and a mixed one as -the marks filling its field. The marks fill the whole width, so every line measures the same -and a block pasted into a message still reads as a grid; reading takes any run of them. +**A field prints what the grid prints in its cell**, which carries the three states across. A value reads as its value, an empty cell as the dots beneath it, and a mixed one as the marks filling its field. The marks fill the whole width, so every line measures the same and a block pasted into a message still reads as a grid. Reading takes any run of them. -**The header is a declaration the body is held to.** It names the grid, the count of rows, and -the span of slots or positions the block stands on, and a body whose lines or fields disagree -with it states no block. The span also carries the alignment a tracker block needs, since the -first slot decides which subcolumn the block opens on. +**The header is a declaration the body is held to.** It names the grid, the count of rows, and the span of slots or positions the block stands on. A body whose lines or fields disagree with it is not a block. The span also carries the alignment a tracker block needs, since the first slot decides which subcolumn the block opens on. -**A note names its voice by list position**, the figure the grid prints, so a block carried to -another project plays whichever voice stands at that position there. A position the project's -list falls short of reads as mixed, which is what the writer already makes of a voice it has -nothing to place. +**A note names its voice by list position**, the figure the grid prints. A block carried to another project therefore plays whichever voice stands at that position there. A position the project's list falls short of reads as mixed, as the writer already treats a voice it has nothing to place. -A field the form has no reading for refuses the whole text, so a parse answers with a block or -with nothing. Digits are read in either case, and transpose and volume are held to the ranges a -row accepts, so text typed by hand lands the values the grid would. +A field the form has no reading for makes the whole text a refusal, so a parse answers with a block or with nothing. Digits are read in either case, and transpose and volume are held to the ranges a row accepts, so text typed by hand lands the values the grid would. ### Which block a paste writes -A copy writes both clipboards, and a paste asks the desktop for its text first: it stands while it -parses as a block for *that* grid, and any other text leaves the grid's own block in hand. So a -block copied in a second instance pastes here, and a copy taken in this one survives whatever -else the desktop picks up afterward. `can_paste_block` asks the same question through a -`ParsedBlockCache`, which reparses only when the text has changed, so opening a menu costs one -string compare. +A copy writes both clipboards, and a paste asks the desktop for its text first. That text stands while it parses as a block for *that* grid. Any other text leaves the grid's own block in hand. A block copied in a second instance therefore pastes here, and a copy taken in this one survives whatever else the desktop picks up afterward. `can_paste_block` asks the same question through a `ParsedBlockCache`, which reparses only when the text has changed, so opening a menu costs one string compare. -The program that owns the desktop's clipboard answers a read in its own time. A paste therefore -writes its block only after the answer arrives. A menu shows Paste based on the last answer it -received, and updates the item when the new answer arrives. +The program that owns the desktop's clipboard answers a read in its own time. A paste therefore writes its block only after the answer arrives. A menu shows Paste based on the last answer it received, and updates the item when the new answer arrives. ## A grid declares its actions once -Where they are shown is decided by whoever asks for them. Each grid builds its whole -action set from one **target** — the cell a gesture is aimed at, paired with the region -that gesture acts on — and three doors resolve that target their own way: - -| Door | Aims at | Anchors a paste at | -|------|---------|--------------------| -| The keyboard | the cursor's cell | the cursor | -| A context menu | the cell it was raised on | the clicked cell | -| The menu bar's **Edit** menu | the cursor's cell | the cursor | - -The region behind a target is `region_at` on the shared input state: the selection when the -cell falls inside it (`Region.covers`), and the cell alone otherwise. So copying one cell -needs no selection made first, and a menu raised inside a selection acts on the whole of it. - -One builder means an action added to a grid appears at every door, and the accelerator -**Edit** prints is the one that grid answers to, since a binding is declared once and every -reader of it reads that entry ([Architecture](../architecture.md), principle 12). - -`EditRouter` (`coordinators/edit/`) is the menu-side counterpart of the `KeyRouter` the -keyboard runs through. Each surface states whether it owns the editing gestures at this -moment — the same predicate its key scope answers with, so the menu offers what the next -press would reach — and the router asks the one that does to build its items into the menu -the bar has opened. It holds no state, resolving the surface on each call, so the menu -states the actions of whoever holds the cursor at the moment it is opened. The bar names the -clipboard four grayed out when no grid answers, which is how a reader working from the menus -learns the commands exist. - -**`Del`** carries two meanings, resolved by whether a selection stands. Two ids cannot share -a combination inside a shortcut category, so this branch is the route; it also matches -tracker convention. +Where they are shown is decided by whoever asks for them. Each grid builds its whole action set from one **target**: the cell a gesture is aimed at, paired with the region that gesture acts on. Three surfaces resolve that target their own way. The keyboard and the menu bar's **Edit** menu both aim at the cursor's cell and anchor a paste there. A context menu aims at the cell it was raised on and anchors a paste at that cell. + +The region behind a target is `region_at` on the shared input state: the selection when the cell falls inside it (`Region.covers`), and the cell alone otherwise. Copying one cell therefore needs no selection made first, and a menu raised inside a selection acts on the whole of it. + +One builder means an action added to a grid appears on every surface. The accelerator **Edit** prints is the one that grid answers to, since a binding is declared once and every reader of it reads that entry ([Architecture](../architecture.md), principle 12). + +`EditRouter` (`coordinators/edit/`) is the menu-side counterpart of the `KeyRouter` the keyboard runs through. Each surface says whether it owns the editing gestures at this moment, with the same predicate its key scope answers with, so the menu offers what the next press would reach. The router asks the surface that does to build its items into the menu the bar has opened. The router holds no state and resolves the surface on each call, so the menu shows the actions of whoever holds the cursor at the moment it is opened. When no grid answers, the bar shows the clipboard commands grayed out, so a reader working from the menus learns they exist. + +**`Del`** carries two meanings, resolved by whether a selection stands. Two ids cannot share a combination inside a shortcut category, so this branch is the route. It also matches tracker convention. ## One gesture, one history entry -Cut, delete and paste each record exactly one entry, whichever door fired them, and none of -them coalesces: a block gesture is already a whole gesture, and folding two consecutive -pastes would hide a repeat the reader performed on purpose. Copy runs outside a transaction, -since it mutates nothing. +Cut, delete and paste each record exactly one entry, whichever surface fired them, and none of them coalesces. A block gesture is already a whole gesture, and folding two consecutive pastes would hide a repeat the reader performed on purpose. Copy runs outside a transaction, since it mutates nothing. -A shift coalesces, because a nudge is a step of one gesture rather than a whole one. The block it -covers is its coalescing target, so a streak over one selection leaves a single step to undo and a -shift after the cursor moves or the selection is reached out starts the next entry. Transpose and -volume count separately, each carrying its own action. +A shift coalesces, because a nudge is a step of one gesture and not a whole one. The block it covers is its coalescing target, so a streak over one selection leaves a single step to undo. A shift after the cursor moves or the selection is reached out starts the next entry. Transpose and volume count separately, each carrying its own action. ## A shape selects to the grid's own edges -`Ctrl+A` and its neighbors select a whole shape at once. Each shape is stated on the input -state as a run of bounds along one axis — slots in the tracker, rows in the order — handed to a -single builder that spans the other axis to the grid's full extent and lands the cursor on the -far corner. The whole frame, a column and a subcolumn are therefore three namings of one -rectangle, as the whole order and a channel row are of the other, and a grid laying out nothing -keeps the selection it had. +`Ctrl+A` and its neighbors select a whole shape at once. The input state gives each shape as a run of bounds along one axis: slots in the tracker, rows in the order. A single builder takes that run, spans the other axis to the grid's full extent, and lands the cursor on the far corner. The whole frame, a column and a subcolumn are therefore three namings of one rectangle, as the whole order and a channel row are of the other. A grid that lays out nothing keeps the selection it had. -The aggregate is an ordinary member of the axis here: selecting the **Voice** column selects a -column the way selecting a channel does, and the **Master** row a row. +The aggregate is an ordinary member of the axis here: selecting the **Voice** column selects a column the way selecting a channel does, and the **Master** row a row. -A press names its shape from the cell the cursor stands on, which is the cell the context menu's -items name too, so a key and an item reach the same rectangle. In the tracker a shape ends at -the frame's last row, so standing one carries the grid to where the cursor landed — the same -reveal a `Shift+End` reach makes. +A press names its shape from the cell the cursor stands on, which is the cell the context menu's items name too, so a key and an item reach the same rectangle. In the tracker a shape ends at the frame's last row, so standing one carries the grid to where the cursor landed, with the same reveal a `Shift+End` reach makes. ## Dragging a range out -Both grids compose one `TableSelection` (`ui/elements/table/selection.py`), which holds what -stands painted and the drag gesture that draws it. The grid states which of its cells the -selection covers, in its own coordinates; the repaint that follows reaches the cells whose -membership changed, marking each through the selectable's own selected state, which the -table's theme colors. A rebuilt table asks for a reset, since the cells a selection stood on -belong to the body that was replaced. - -Both panels read the cell under a held pointer off their own geometry, because DearPyGui -reports no hover for the cells a held pointer passes over. A drag carried past an edge -reads as the edge, so it selects up to it. - -The tracker's row lookup is arithmetic: it takes the first row's top edge and divides by -`layout.tracker.row_height`. That holds only while the rows are evenly pitched, which is -what `CellPadding.y = 0` and `ItemSpacing.y = 0` in `theme/tables/pattern.yaml` are for. -A vertical padding there would drift the lookup further down the grid. The order's -position lookup is arithmetic in the same way, taking its pitch from the first two -columns; its channel lookup walks the rows, because the master row stands apart from the -channels beneath it. +Both grids compose one `TableSelection` (`ui/elements/table/selection.py`), which holds what stands painted and the drag gesture that draws it. The grid says which of its cells the selection covers, in its own coordinates. The repaint that follows reaches the cells whose membership changed and marks each through the selectable's own selected state, which the table's theme colors. A rebuilt table asks for a reset, since the cells a selection stood on belong to the body that was replaced. + +Both panels read the cell under a held pointer off their own geometry, because DearPyGui reports no hover for the cells a held pointer passes over. A drag carried past an edge reads as the edge, so it selects up to it. + +The tracker's row lookup is arithmetic. It measures from the first row's top edge and divides by the row height, which holds while the rows are evenly pitched. That is what the pattern table's zero vertical cell padding and item spacing are for: a vertical padding there would drift the lookup further down the grid. The order's position lookup is arithmetic in the same way and takes its pitch from the first two columns. Its channel lookup walks the rows, because the master row stands apart from the channels beneath it. ### A drag past the edge carries the view -A pointer held past the cells on screen travels the grid under it, so a selection reaches -further than the viewport holds. `grid/scroll/` states this in three pieces: a `ScrollAxis` -naming the one DearPyGui axis a table scrolls along and the pointer coordinate that runs past -its edges, a `TravelBand` saying where the cells stand along that axis, and the `DragTravel` -that reads the two each frame. The tracker travels vertically and the order horizontally, both -from the same class. - -Three rules make the travel feel like one gesture: - -- **The pointer report drives it.** A held pointer keeps reporting wherever it is carried to, - including past the window, so the travel runs off the same report the drag itself reads. -- **The frame's own duration paces it**, so the same stretch of grid passes under the pointer - however fast the frames arrive. The pace answers how far past the edge the pointer stands, - rising from a floor to a ceiling over a few cells' overshoot: a nudge creeps, a reach covers - the grid. -- **Each step is added to the offset last issued.** A table reports the scroll it was drawn - with rather than the one just set, so a travel reading it back would re-issue an offset it - has already reached. It rests as soon as the pointer stands within the band again, at the - press that opens the next gesture, and on a rebuild — and the travel that follows sets out - from the offset the grid is drawn with. +A pointer held past the cells on screen travels the grid under it, so a selection reaches further than the viewport holds. `grid/scroll/` holds the travel. A grid says which axis it scrolls along and where its cells stand along it, and one class reads the two each frame. The tracker travels vertically and the order horizontally, from that same class. + +These rules make the travel feel like one gesture: + +- **The pointer report drives it.** A held pointer keeps reporting wherever it is carried to, including past the window, so the travel runs off the same report the drag itself reads. +- **The frame's own duration paces it**, so the same stretch of grid passes under the pointer however fast the frames arrive. The pace depends on how far past the edge the pointer stands. It rises from a floor to a ceiling over a few cells' overshoot: a nudge creeps, and a reach covers the grid. +- **Each step is added to the offset last issued.** A table reports the scroll it was drawn with and not the one just set, so a travel that read it back would re-issue an offset it has already reached. The travel rests as soon as the pointer stands within the band again, at the press that opens the next gesture, and on a rebuild. The travel that follows sets out from the offset the grid is drawn with. ## Accepted limitations -- **A rebuilt table has no selection.** Both grids reconstruct their input state on - rebuild, so following playback and the rebuild after a growing paste leave the cursor - and drop the selection. The rows a region named belong to the body that was replaced. -- **The selection stays put after a paste** rather than becoming the pasted footprint. -- **A note crosses a project by whichever route it took.** The in-app slot survives a project - close, because it must survive `on_project_replaced`, which fires on every undo, and it names - its voice by id: a note whose voice the project in place lacks is left out of the write, and - the target keeps what it had. The clipboard's text names a list position instead, so the same - note pasted through it plays whichever voice stands at that position. Transpose and volume - are exact by either route. -- **A drag past the edge and the followed playhead both write the scroll.** With **Follow rows** - on during playback, `_reveal_playing_row` carries the sounding row to the head of the band - while a held pointer travels the grid, so the two take turns each frame. +- **A rebuilt table starts without a selection.** Both grids reconstruct their input state on rebuild, so following playback and the rebuild after a growing paste leave the cursor and drop the selection. The rows a region named belong to the body that was replaced. +- **The selection stays put after a paste** and does not become the pasted footprint. +- **A note crosses a project by whichever route it took.** The in-app slot survives a project close, because it must survive `on_project_replaced`, which fires on every undo, and it names its voice by id. A note whose voice the project in place lacks is left out of the write, and the target keeps what it had. The clipboard's text names a list position instead, so the same note pasted through it plays whichever voice stands at that position. Transpose and volume are exact by either route. +- **A drag past the edge and the followed playhead both write the scroll.** With **Follow rows** on during playback, the followed playhead carries the sounding row to the head of the band while a held pointer travels the grid, so the two take turns each frame. diff --git a/docs/development/application/stems.md b/docs/development/application/stems.md new file mode 100644 index 000000000..93cd594c5 --- /dev/null +++ b/docs/development/application/stems.md @@ -0,0 +1,92 @@ +# Stems in the application + +This document covers what the application does with a reconstruction built from several stems: how it loads and names the recorded stems, what the Stems card offers, and what an edit or a removal does to the per-frame record. Consult it when changing an edit path, the record, or what the card offers. + +The assignment that writes the record is [Stems reconstruction](../../concepts/stems.md). [Architecture](../architecture.md) describes the layers this code sits in. + +Five terms recur. The **record** is the per-frame account of which recording plays each frame of each channel. A frame's **owner** is the recording the record names for it. A frame **rests** when nobody holds it, and the record then names the resting stem id. An **authored** frame is one the reader wrote by hand, and the record names the authored stem id. A **level** is a rank in the hierarchy the conversion used. + +## The recorded stems + +A stems reconstruction records one source per entry, naming its recording and the file it was read from. The application reads the locations through `source_paths`: one path for a single source, a tuple for stems, and empty once the reconstruction is detached from its origin. The names stay whatever happens to the locations, so a detached document still says which recordings it was built from. + +Opening the document loads the recorded stems through `load_stems`, the same call the conversion loads them with. Each stem therefore carries the level it holds in the mix, and a stem heard on its own sounds at that level. The mix of them is the original audio the source toggle and the waveform offer, computed fresh on every load. A recorded stem that is absent or unreadable on this machine follows the single-source rule: the whole original is unavailable, the approximation stands on its own, and the application names the first missing path in its dialog. + +The document's name follows the naming rules in `sampletones_core.reconstructions.naming`, applied to the recorded paths in order. A single source names the document after the file's stem. Several stems that share one directory name it after that directory. Paths that share no directory fall back to the `.stn` filename. + +The Stems card names every recorded path, one row per stem. Each row has its own full-path tooltip and reveals its recording on a click. The Audio source panel keeps the reconstruction's own file and the choice between the two waveforms. Locating reveals every recorded path at once, in one window with every stem selected where the file manager supports that, and in one window per directory otherwise. + +## The stems card + +The reconstruction tab's Stems card turns the recorded assignment into a listener the user can steer. It draws the same list the converter's card draws: each row is one stem under the level it was picked on, named by its recording, with a leading master box and a colored box on every channel the stem holds frames on. A setup line above the rows names the assignment's hierarchy mode, and a **Collapse levels** toggle draws every row in one table where the banding is in the way. Ticking a box admits that stem's frames on that channel to everything the tab plays and exports, and unticking silences them. + +### Principles of the card + +1. **Selection filters what plays, shows what it filters, and scopes what is edited.** A ticked set projects the document and does not mutate it. The waveform shows the ticked frames alone. The reconstruction toggle plays them mixed, and original playback plays the recordings heard anywhere, mixed. The instruments panel draws the envelopes of that same part beside the figures measuring it, and both WAV export and instrument export write what stands on screen. Each answer derives from the recorded per-channel assignment, so a stem heard on one channel keeps its samples there and stays quiet on the next. The same set says which frames an instrument edit writes (see [Editing a stems reconstruction](#editing-a-stems-reconstruction)). + + Two rules shape the reading. **A filtered reading shows the frames it leaves out as silence in place**, so the envelopes, the waveform and the record line up column for column. **It ends at the last frame the reader hears**, so a channel whose sound the reader's choice took away reads as standing by: its plot is empty, its figures are at nothing and its instrument is written nowhere. A rest answers to no recording, so every reader hears it. A channel written down to rests alone stays in play, and the figures beside a plot name the bytes the export writes. Together these rules make what a reader sees, hears, edits and exports one and the same part of the document. +2. **A box appears where the choice reaches something.** A stem draws a box on a channel exactly where the record gives it a frame there, so every box the card offers changes what is heard. A stem the picker never chose offers none, and its row reads as holding no frames. The frames a reader wrote by hand answer to no recording, so they gather in a row of their own that reads and behaves like any other. +3. **Every stem starts heard everywhere it holds frames.** A freshly opened stems reconstruction ticks every box, so the waveform and the original are the unfiltered document. +4. **The global channel choice takes precedence.** A channel switched off for the whole reconstruction mutes its column and leaves every value where the reader put it, so switching the channel back on restores the per-stem choice intact. The two compose by construction: the global choice filters the partials, and the stems choice filters the approximations. +5. **The selection follows the open document.** The card lives with the reconstruction it describes. Opening a document seeds the rows and the ticked boxes, a regenerated reconstruction keeps what the reader chose and ticks the channels a stem newly reaches, and closing the document empties the card. +6. **Listening choices stay out of the document.** The ticked set is session state, like every choice that shapes what is heard (see [Playback](playback.md)). Saving the reconstruction records the assignment and not the selection. The banding works the same way: collapsing the levels changes how the card draws and never what it describes. +7. **Removing a recording edits the document.** Where a box steers listening, the remove button rewrites what is described. The change is asked about first and recorded in the project history, and [Editing a stems reconstruction](#editing-a-stems-reconstruction) says what it releases. A reconstruction holds at least one recording, so the last row standing keeps its button disabled. +8. **The ribbon shows what the record holds.** Under the waveform, a lane per channel in play divides into the stretches one recording holds throughout, each painted in that recording's color. A recording takes its color from the place it holds on the record, so one recording reads alike wherever it is drawn. The lanes appear only where more than one owner is in play, either a recording the record names or the row the frames a reader wrote gather under, since a document with a single owner has nothing to tell apart. A stretch reads solid where the reader hears the recording holding it, faded where the reader left it out, and in the surface's own ground where the frames rest. It names its owner whether or not it is listened to. The instruments panel paints the same stretches in a band beneath each dimension's bars. + +### What the code guarantees + +A filtered mix keeps every array at its unfiltered length, so it aligns with the unfiltered one sample for sample. The filter zeroes the unselected frames per channel before mixing (`filter_approximations`), and the original mix covers the recordings of the stems heard on any channel. The panel logic holds the channels each stem is heard on and re-answers the stems view model, the waveform and the audio data whenever the choice changes. A reconstruction that records one source presents a single row for its recording, and one that records no source shows the card's empty state. + +Removal runs through `without_stem`, which returns a fresh reconstruction holding what the rules in [Editing a stems reconstruction](#editing-a-stems-reconstruction) leave. The tab coordinator hands the result on as a `ReconstructionEdit`, the payload both a regenerated instrument and a removed recording travel as. One path therefore rebinds the open document and records the edit against the project history. + +## Editing a stems reconstruction + +A conversion answers, per channel and frame, which recording plays there. Everything a reader does to the document afterward stands on that answer: the instruments panel rewrites what a channel plays, the stems card chooses what is heard, and the remove button takes a recording out. This section describes the account of ownership all three keep, and the rules each gesture follows. + +### Principles of editing + +1. **A frame has exactly one owner.** Every frame of a channel in play is held by one recording, by the reader's own hand, or by nobody. The hardware reads one instruction per channel per frame, so the channel itself allows only this, and it makes the per-frame record a partition of the channel's frames. The resting stem id names the frames nobody holds, and the authored stem id names the frames the reader wrote. +2. **Rest and silence name the same frames.** A frame rests exactly when its instruction is silent, which the conversion establishes and every later gesture keeps. A reader looking at a silent frame and a reader looking at the record therefore learn the same thing, and the rules below follow from it. +3. **An edit rewrites what a frame plays and leaves who plays it.** Ownership answers a question an edit does not ask, so a frame carries its owner through any change to its instruction. A frame takes an owner by coming into play and releases it by falling silent. A reader can therefore shape a recording's part while the document keeps its account of where that part came from. +4. **A reconstruction records what is played and reads what is heard.** The document holds the instruction each channel plays per frame, the recording behind each of those frames, the setup they were chosen under and the working level. Its sound is read from those through the generators, as the instructions stand. That one answer serves the waveform, playback, an export and the mixed approximation alike. A drive settles which instruction a conversion records and is read no further. Every gesture below is therefore complete once it has settled the frames. +5. **What is heard is what is edited.** The channels a recording is ticked on are one choice serving two readings: the frames the waveform draws, and the frames an edit writes. A recording switched off on a channel reads there and stays as it stands, so a reader reaches one recording's part at a time through the card already in front of them. +6. **The setup is the conversion's, and a removal alone rewrites it.** The entries, the levels, the channels each recording may occupy, its bends, its drives and its count record how the document was made. An edit leaves all of them as they stand. Taking a recording out is the one gesture that changes them. +7. **Detaching drops where a recording lives and keeps who played what.** A recording's name and the frames it holds belong to the document, and its location belongs to this machine. The record keeps both, so detaching lets each location go and keeps every name. A reconstruction embedded in a project therefore keeps a working stems card. + +### What the record holds + +The per-frame record answers for every channel in play, and these hold after every gesture: + +- A channel in play has one owner per frame of its stream, each naming a recorded entry, the resting stem id or the authored stem id. +- A channel standing by has no stream and no record at all. +- A frame rests exactly where its instruction is silent. +- A frame a recording holds lies on a channel that recording's settings occupy, and an authored frame answers to no settings. +- An entry that holds no frame anywhere stays on the record, and its row reads as holding none. + +### What an edit carries + +An edit hands one channel a fresh set of envelopes, which become that channel's stream. Each frame's owner follows from the frame it was and the frame it becomes: + +| the frame | becomes | +|---|---| +| sounding before and after | its owner, unchanged | +| sounding, edited silent | resting | +| resting, edited into play | authored | +| resting, edited silent | resting | +| written past the end of the stream | authored where it sounds, resting where it stays silent | +| dropped from the end of the stream, within the scope | gone, together with its ownership | +| dropped from the end of the stream, outside it | standing, in what it plays and who plays it | + +Rest and silence naming the same frames makes the table total. A frame's owner before the edit already says whether it sounded, so the first four lines cover every frame the edit keeps at its position. + +An edit shortens a channel as far as its scope reaches. The stream therefore runs through the last frame standing outside that scope. The frames the edit did reach rest along the way, and a stream edited down to no frame leaves its channel standing by wherever the scope covered every one of them. A channel written back into play comes back wholly authored. The setup, the recorded sources, the identifier, the configuration and the working level stand throughout. + +**The scope an edit writes in** follows principle 5: a frame accepts a gesture where its owner is ticked on that channel, and where it rests. The frames a reader wrote answer to a row of their own, so they are ticked and reached like any recording's. A rest belongs to no row, which lets a gesture write a note into silence. Every other frame draws dimmed and reads as it stands. The scope is the same selection the waveform filter reads, so one state answers both, and a reader narrowing what they hear narrows what they change with it. + +### What a removal releases + +Taking a recording out (`without_stem`) releases the frames it held. Each takes its channel's silent instruction and the resting stem id, and a channel the removal empties stands by. The entry and its source leave the record, and a level the removal empties collapses. The ids of the recordings that stay are left alone, so the record and a reader's selection both stay valid. A reconstruction holds at least one recording, so the last one standing keeps its place. + +A frame that stays keeps the instruction it played and the recording that held it. Its samples follow from principle 4: a channel the removal reached is read afresh, so its oscillator runs through the silence the removal left and not through the notes it took away. + +An edit made before the removal changes nothing about it: the channel carries its record whatever was written into it, so the frames the recording held are released the way they would have been. The frames the reader authored answer to no recording and stand through every removal. diff --git a/docs/development/application/undo.md b/docs/development/application/undo.md index 1321705ed..fe97a4fdd 100644 --- a/docs/development/application/undo.md +++ b/docs/development/application/undo.md @@ -1,95 +1,38 @@ # History & Undo -This document describes the undo/redo subsystem of `sampletones_application`. The engine lives in `logic/history/` and is owned by `HistoryManager`; coordinators integrate with it as described in `docs/development/architecture.md`. Undo/redo is session-scoped and upholds two invariants: +This document describes the undo/redo subsystem of `sampletones_application`. Consult it when adding a gesture that changes project state, or when changing what the history records. The engine lives in `logic/history/` and is owned by `HistoryManager`. Coordinators integrate with it as described in [`architecture.md`](../architecture.md). Undo/redo is session-scoped and upholds two invariants: -1. **Completeness** — every mutation of project state belongs to the history. -2. **Reversibility determinism** — any composition of undos and redos that returns - the cursor to an index reproduces that index's exact state. +1. **Completeness.** Every mutation of project state belongs to the history. +2. **Reversibility determinism.** Any composition of undos and redos that returns the cursor to an index reproduces that index's exact state. ## Engine: snapshot + cursor -`HistoryManager` holds an ordered list of whole-project snapshots and a cursor; -the live project always equals a restoration of `entries[cursor]`. Undo and redo -move the cursor and reinstall the snapshot there — they never mutate a stored -snapshot, so reversibility determinism holds by construction. Restore installs a -fresh copy through `ProjectController.replace_project`, which fires -`on_project_replaced` to rebuild the tabs exactly as loading a project does. - -A snapshot (`snapshot_project`) deep-copies the light structure (song, settings, -metadata, sample shells) but **shares each `Reconstruction` by reference**. -Reconstruction edits are copy-on-write: `RegenerationService` emits a *new* -reconstruction and the apply path installs it via -`ProjectController.replace_sample_reconstruction`, so a shared reconstruction -never mutates in place and snapshots never duplicate the multi-megabyte audio -arrays. Producing the fresh reconstruction deep-copies the edited one once, on -the regeneration worker's background thread. - -## Grouping vs. detection - -- **Grouping — coordinators.** Each state-changing coordinator intent runs inside - `HistoryManager.transaction(HistoryAction.X)` (the sequencer wraps its hooks via - `_undoable`). All controller calls a gesture makes collapse into one entry; - nested transactions coalesce. A transaction may also carry a *coalesce key* - naming the gesture's target (a grid cell, a sample, a module setting): - consecutive commits sharing the same action and key replace the top entry - instead of appending, so a continuous interaction — a graph drag, repeated - edits of one cell — records a single entry. Any undo, redo, or jump breaks - the run, so a state the user navigated to is always preserved. `_undoable` - opens `ProjectController.batch()` inside the transaction, so one gesture is - one entry and one round of view notifications alike. -- **Detection — the controller.** `ProjectController._touch()` fires `on_mutation` - on every fine-grained mutation, as it lands — a batch defers the view - notifications and the dirty stamp, leaving this signal immediate so the check - below sees each mutation inside the transaction that caused it. - `HistoryManager.handle_mutation` counts those - inside a transaction and rejects any that occur outside one: under strict - deployment it raises `UntrackedMutationError`; otherwise it self-heals by - recording the mutation as its own entry. This makes completeness a checkable - property. Under strict deployment each committed snapshot also carries a - fingerprint, and every restore verifies the reproduced project matches it. - Capture-time fingerprints memoize each reconstruction's hash by object - identity (copy-on-write keeps the content fixed for the object's lifetime), - collapsing the per-gesture cost to the light structure; restore-time - verification always hashes fresh, so an in-place mutation of shared state is - caught rather than masked by the memo. +`HistoryManager` holds an ordered list of whole-project snapshots and a cursor. The live project always equals a restoration of `entries[cursor]`. Undo and redo move the cursor and reinstall the snapshot there. They never mutate a stored snapshot, so reversibility determinism holds by construction. A restore installs a fresh copy through `ProjectController.replace_project`, which fires `on_project_replaced` to rebuild the tabs exactly as loading a project does. + +A snapshot (`snapshot_project`) deep-copies the light structure (song, settings, metadata, sample shells) and **shares each `Reconstruction` by reference**. Reconstruction edits are copy-on-write. `RegenerationService` emits a *new* reconstruction, and the apply path installs it via `ProjectController.replace_sample_reconstruction`. A shared reconstruction therefore never mutates in place, and snapshots never duplicate the large audio arrays. Producing the new reconstruction deep-copies the edited one once, on the regeneration worker's background thread. + +## Grouping and detection + +**Grouping happens in the coordinators.** Each state-changing coordinator intent runs inside `HistoryManager.transaction(HistoryAction.X)`. The sequencer wraps its hooks with `_undoable`, which also opens `ProjectController.batch()` inside the transaction. Every controller call a gesture makes collapses into one entry, and one gesture is one round of view notifications too. Nested transactions merge into the outermost one. + +A transaction can carry a *coalesce key* that names the gesture's target, such as a grid cell, a sample or a module setting. Consecutive commits with the same action and key replace the top entry and do not append. A continuous interaction, such as a graph drag or repeated edits of one cell, therefore records a single entry. Any undo, redo or jump ends the run, so a state the user navigated to is always preserved. + +**Detection happens in the controller.** `ProjectController._touch()` fires `on_mutation` on every fine-grained mutation as it lands. A batch defers the view notifications and the dirty stamp and leaves this signal immediate, so the check below sees each mutation inside the transaction that caused it. `HistoryManager.handle_mutation` counts the mutations inside a transaction and rejects any that occur outside one. Under strict deployment it raises `UntrackedMutationError`. Otherwise it heals by recording the mutation as its own entry. This makes completeness a checkable property. + +Under strict deployment each committed snapshot also carries a fingerprint, and every restore verifies that the reproduced project matches it. Capture-time fingerprints memoize each reconstruction's hash by object identity, since copy-on-write keeps the content fixed for the object's lifetime. That reduces the per-gesture cost to the light structure. Restore-time verification always hashes fresh, so an in-place mutation of shared state is caught and not masked by the memo. ## Save point and lifecycle -The manager records the cursor of the last successful save (wired from -`ProjectController.on_saved`); a restore that lands exactly on that index -reinstates the on-disk content, so the session reports the document clean -again. A commit that truncates the saved entry away — or budget eviction that -drops it — invalidates the save point, and the session stays dirty until the -next save. Coalescing always preserves the saved entry by appending. The stack -follows the project lifecycle: an open project seeds a baseline entry, and -closing every project empties the stack, so the panel reports no history. +The manager records the cursor of the last successful save (wired from `ProjectController.on_saved`). A restore that lands exactly on that index reinstates the on-disk content, so the session reports the document clean again. A commit that truncates the saved entry away, or budget eviction that drops it, invalidates the save point, and the session stays dirty until the next save. Coalescing always preserves the saved entry by appending. + +The stack follows the project lifecycle. An open project seeds a baseline entry, and closing every project empties the stack, so the panel reports no history. ## History detail rendering -Committed entries are language-independent. An entry stores its action as a -`HistoryAction` enum member and its detail as data segments; language-managed -words inside a detail (e.g. a loop's on/off state) are stored as -`HistoryDetailWordSegment` keys. Action labels and word segments alike resolve -through `LanguageManager` when the history view model is built, so switching -the language re-renders past entries correctly. +Committed entries are language-independent. An entry stores its action as a `HistoryAction` enum member and its detail as data segments. Language-managed words inside a detail, such as a loop's on/off state, are stored as `HistoryDetailWordSegment` keys. Action labels and word segments alike resolve through `LanguageManager` when the history view model is built, so switching the language re-renders past entries correctly. ## Configuration -The entry budget is a persisted user preference -(`ApplicationConfig.history.budget`, default 500, lower bound 1). Strict -checking and log level are deployment knobs -(`application/deployment.yaml` → `DeploymentConfig`); the deployment model is -authoritative from YAML with no field defaults. The history panel renders a -window of `layout.sequencer.history.max_rendered_entries` rows around the -cursor and repaints rows in place via an index-keyed diff. - -**Strict checking is on where the code is written.** `deployment.yaml` carries the development -values, so an edit path reaching the project outside a transaction raises -`UntrackedMutationError` at once — in a development run and in the test suite alike, since -several tests build the whole application and read that file. A user build takes the opposite -values from `scripts/runtime_hooks/release_environment.py`, so a gap that reaches a release is healed into an -`UNTRACKED` entry rather than shown to the user. The gap therefore surfaces where it can be -fixed and stays quiet where it cannot. - -Standalone reconstruction documents (a reconstruction loaded from disk that is not -a project sample) will gain their own history later, reusing the same engine. +The entry budget is a persisted user preference (`ApplicationConfig.history.budget`). Strict checking and log level are deployment settings (`application/deployment.yaml` → `DeploymentConfig`), and the deployment model takes every value from the YAML with no field defaults. The history panel renders a window of rows around the cursor (`layout.sequencer.history.max_rendered_entries`) and repaints rows in place through an index-keyed diff. + +**Strict checking is on where the code is written.** `deployment.yaml` has the development values, so an edit path that reaches the project outside a transaction raises `UntrackedMutationError` at once. That holds in a development run and in the test suite alike, since several tests build the whole application and read that file. A user build takes the opposite values from `scripts/runtime_hooks/release_environment.py`. A gap that reaches a release is healed into an `UNTRACKED` entry and is not shown to the user. The gap therefore surfaces where it can be fixed and stays quiet where it cannot. diff --git a/docs/development/application/vocabularies.md b/docs/development/application/vocabularies.md index 1181ff4ac..57fb0b482 100644 --- a/docs/development/application/vocabularies.md +++ b/docs/development/application/vocabularies.md @@ -1,10 +1,10 @@ # Identifier Vocabularies Two vocabularies name things across `sampletones_application`: the keys every user-visible string is -looked up by, and the identifiers DearPyGui knows a widget by. Both are spelled by a grammar, both -are held to the source whole-tree by a pre-commit hook, and both keep in one place a fact that would -otherwise be restated at every use. Consult this document when adding a string the reader sees, or a -widget another module reaches. +looked up by, and the identifiers DearPyGui knows a widget by. Both are spelled by a grammar. A pre-commit +check, called a hook here, holds each one to the source across the whole tree. Each keeps in one place a +fact that would otherwise be restated at every use. Consult this document when adding a string the reader +sees, or a widget another module reaches. The design truths it realizes are principles 8 and 9 of [`architecture.md`](../architecture.md): all display text comes from `LanguageManager`, and `tags/` holds only DPG identifiers. This document @@ -22,37 +22,36 @@ Every user-visible string is looked up on `LanguageManager` by the key the langu page.panel.text_type.element ``` -The first three segments name members of `Page`, `Panel`, and `TextType` (`categories/hierarchy.py`); -the element segment names a member of an element enum, which is any enum deriving from -`AbstractElement`. An element enum is found by what it derives from, so one naming a panel's own -widgets lives with the other panel vocabularies under `categories/elements/`, while one naming a -domain's gestures — `HistoryAction` — lives beside that domain and serves as both the value the -domain records and the element its label is looked up by. +The first three segments name members of `Page`, `Panel` and `TextType` (`categories/hierarchy.py`). The +element segment names a member of an element enum, which is any enum deriving from `AbstractElement`. An +element enum is found by what it derives from. One naming a panel's own widgets lives with the other panel +vocabularies under `categories/elements/`. One naming a domain's gestures, such as `HistoryAction`, lives +beside that domain and serves as both the value the domain records and the element its label is looked up +by. -`en.yaml` is a flat map keyed exactly this way, so the dotted string is the lookup form — -`language_manager["global.dialog.label.ok"]` — and a reader holds a key against the language file by -eye. `categories/key/` owns the grammar: `validate_text_key` checks every key the file holds at load -time, and a lookup that misses raises `MissingTextError` naming the key and the file. This makes the -text system the single source of truth and enables future localization. Log messages are -developer-facing and exempt. +`en.yaml` is a flat map keyed exactly this way, so the dotted string is the lookup form, +`language_manager["global.dialog.label.ok"]`, and a reader holds a key against the language file by eye. +`categories/key/` owns the grammar. `validate_text_key` checks every key the file holds at load time, and a +lookup that misses raises `MissingTextError` naming the key and the file. Log messages are developer-facing +and exempt. ### Text resolves where it is displayed -A class that reads text holds the manager as `self._language_manager`, assigned in its own -`__init__`, and looks each string up at the point of use, so a language change takes effect on the -next read. Where the same text is read at more than one site in a class, one named binding serves -them all and the reads stay in step. +A class that reads text holds the manager as `self._language_manager`, assigned in its own `__init__`, and +looks each string up at the point of use, so a language change takes effect on the next read. Where the +same text is read at more than one site in a class, one named binding serves them all and the reads stay in +step. ### The forms a lookup takes -A key assembled at runtime passes its four members instead — -`language_manager[Page.SEQUENCER, Panel.ORDER, TextType.LABEL, element]` — with the variable part -annotated as the concrete element enum it carries (`SequencerOrderElements`, `DialogElements`). That -annotation is what keeps the key checkable: the `language-keys` hook expands it to the enum's -members and holds every key it reaches against the language file. +A key assembled at runtime passes its four members instead, as in +`language_manager[Page.SEQUENCER, Panel.ORDER, TextType.LABEL, element]`. The variable part is annotated as +the concrete element enum it carries (`SequencerOrderElements`, `DialogElements`). That annotation keeps +the key checkable: the `language-keys` hook expands it to the enum's members and holds every key it reaches +against the language file. -A lookup therefore states its key in one of the three forms the hook reads values from: as literals, -as annotated members, or as a conditional between two literal keys. +A lookup therefore states its key in one of the forms the hook reads values from: literals, annotated +members, or a conditional between two literal keys. ```python language_manager[ @@ -66,24 +65,25 @@ language_manager[ ### `compose_tag` is the one composer -`tags/compose.py` owns `TAG_SEPARATOR` and the joiner; every tag reaches its final spelling through -it. Each part is lowercased and its whitespace runs become single underscores, so a tag built from a -runtime name — a sample title, a layer label — reads the same however that name arrives cased or -spaced, and a part already holding a composed tag contributes its own segments, which is how a child -tag extends its parent. Fragments hold bare segments (`SUF_GRAPH_PLOT = "plot"`) and gain separators -only from the joiner, so a fragment reads as the segment it names and either end composes onto it. +`tags/compose.py` owns `TAG_SEPARATOR` and the joiner, and every tag reaches its final spelling through it. +Each part is lowercased and its whitespace runs become single underscores, so a tag built from a runtime +name, such as a sample title or a layer label, reads the same however that name arrives cased or spaced. A +part that already holds a composed tag contributes its own segments, which is how a child tag extends its +parent. Fragments hold bare segments (`SUF_GRAPH_PLOT = "plot"`) and gain separators only from the joiner, +so a fragment reads as the segment it names and either end composes onto it. ### A runtime name carries its identity beside it -Because the composer reads two names that differ only in case or spacing as one segment, a tag built -from a name a user gave — a file path, a project title — carries `identity_part(*parts)` beside the -name. The part is a short digest of the name exactly as it arrived, so `Kick.wav` and `kick.wav` -name widgets of their own, and the readable name stays in the tag so a DearPyGui error still says -which row it is about. `tags/compose.py` states the rule once over -`sampletones_shared.utils.hashing.identity_digest`, which joins the parts on a separator no name -carries; `ui/elements/stems/tags.py` and `ui/elements/tree/tag.py` both read it. A widget keyed by -anything a user names needs it — DearPyGui refuses a duplicate alias, so two names arriving at one -tag break the draw rather than crossing quietly. +The composer reads two names that differ only in case or spacing as one segment. A tag built from a name a +user gave, such as a file path or a project title, therefore carries `identity_part(*parts)` beside the +name. The part is a short digest of the name exactly as it arrived, so `Kick.wav` and `kick.wav` name +widgets of their own. The readable name stays in the tag, so a DearPyGui error still says which row it is +about. + +`tags/compose.py` states the rule once over `sampletones_shared.utils.hashing.identity_digest`, which joins +the parts on a separator no name carries. `ui/elements/stems/tags.py` and `ui/elements/tree/tag.py` both +read it. A widget keyed by anything a user names needs it: DearPyGui refuses a duplicate alias, so two names +arriving at one tag break the draw. ### A whole tag is a `TagName` @@ -96,7 +96,7 @@ TAG_MAIN_EXPLORER_TREE = TagName( ) # main.explorer.tree ``` -The spelling is `page[.panel].widget[.element]` — `Panel.IMPLICIT` names a widget belonging to no -panel, and an element repeating its panel's name is carried by the panel segment alone. A constant's -name is its composed tag upper-cased with each separator turned into an underscore, behind the -`TAG_` prefix, so reading either one states the other; the `tag-names` hook holds the two together. +The spelling is `page[.panel].widget[.element]`. `Panel.IMPLICIT` names a widget that belongs to no panel, +and an element that repeats its panel's name is carried by the panel segment alone. A constant's name is +its composed tag upper-cased, with each separator turned into an underscore, behind the `TAG_` prefix. +Reading either one therefore gives the other, and the `tag-names` hook holds the two together. diff --git a/docs/development/architecture.md b/docs/development/architecture.md index 049ba753f..fb6da5f54 100644 --- a/docs/development/architecture.md +++ b/docs/development/architecture.md @@ -1,14 +1,14 @@ # Application Architecture -This document describes the design of `sampletones_application` — the GUI front-end of _SampleToNES_. It is prescriptive: it states the contracts each layer must honor, in the form they are enforced, and the rationale behind them. Use it as the reference when deciding where new code belongs. +This document describes the design of `sampletones_application`, the GUI front-end of _SampleToNES_. It is prescriptive: it states the contracts each layer must honor, in the form they are enforced, and the reasons behind them. Use it when deciding where new code belongs. -Concrete classes and modules appear throughout as **examples** that anchor a rule; the rules bind every instance, named or not. Known deviations from these contracts are tracked in `docs/development/bugs-and-todos.md`. Coding-level rules live in `docs/development/guidelines.md`; the undo subsystem has its own design document, `docs/development/application/undo.md`, the audio transport has `docs/development/application/playback.md`, the reconstruction browser has `docs/development/application/browser.md`, the YAML configuration package has `docs/development/application/config-organization.md`, how a long operation says how far it has come has `docs/development/progress.md`, the keyboard and the actions it reaches have `docs/development/application/keyboard.md`, the identifier vocabularies have `docs/development/application/vocabularies.md`, colors and palettes have `docs/development/application/palette.md`, and the packages the repository divides into have `docs/development/packages.md`. +Concrete classes and modules appear as **examples** that anchor a rule. The rules bind every instance, named or not. [`bugs-and-todos.md`](bugs-and-todos.md) tracks known deviations from these contracts. [`guidelines.md`](guidelines.md) has the rules for code and docstrings. [`docs/index.md`](../index.md) lists the subsystems that have a design document of their own. --- ## Overview -`sampletones_application` is a [DearPyGui](https://github.com/hoffstadt/DearPyGui) application that exposes the `sampletones_core` audio-reconstruction engine through a multi-tab GUI. Four layers with clearly bounded responsibilities structure the code — **UI** (widget construction), **view models** (immutable projections), **logic and services** (domain state and background work), and **coordinators** (orchestration) — with dependencies flowing in one direction only. A single composition root (`Application`) constructs and wires all components at startup. +`sampletones_application` is a [DearPyGui](https://github.com/hoffstadt/DearPyGui) application that exposes the `sampletones_core` audio-reconstruction engine through a multi-tab GUI. The code is structured in layers with bounded responsibilities: **UI** (widget construction), **view models** (immutable projections), **logic and services** (domain state and background work) and **coordinators** (orchestration). Dependencies flow in one direction only. A single composition root (`Application`) constructs and wires all components at startup. ```mermaid graph TD @@ -39,328 +39,203 @@ These principles govern every structural decision in the codebase. ### 1. Layering: dependencies flow inward -Each layer imports only from the layers below it. Coordinators, at the top, reach every layer they orchestrate. The UI layer knows only view models and shared utilities. Logic owns domain state and produces view models. Services, at the bottom of the application stack, know only the core libraries and thread-safe utilities — a service is driven through a logic-side `Protocol` and reports through its result-contract types, so even logic reaches a service only through inversion. +Each layer imports only from the layers below it. Coordinators, at the top, reach every layer they orchestrate. The UI layer knows only view models and shared utilities. Logic owns domain state and produces view models. Services, at the bottom of the application stack, know only the core libraries and thread-safe utilities. A service is driven through a logic-side `Protocol` and reports through its result-contract types, so even logic reaches a service only through inversion. -The load-bearing prohibitions: nothing in `logic/` or `services/` imports `ui/` or `coordinators/`, and `services/` imports neither `logic/` nor `view_model/`. The authoritative import matrix is the **May import / Must not import** pair in each Layer Reference section below; the boundary check enforces it (see Enforcement). +The boundaries that carry the weight: nothing in `logic/` or `services/` imports `ui/` or `coordinators/`, and `services/` imports neither `logic/` nor `view_model/`. `sampletones_config/boundaries/rules.yaml` declares which layer may reach which, one rule per layer, and the boundary check holds the tree to it (see Enforcement). ### 2. DPG stays in the visual layers -Calls into `dearpygui` are confined to `ui/`, `shell.py`, and the narrow coordinator surface defined in the Layer Reference. The `logic/`, `services/`, `view_model/`, and `config/` layers remain DPG-free so they can be instantiated and tested without a running GUI context. This extends past `import dearpygui`: the dpg-bound helpers (`DialogsRenderer`, the `dpg_*` wrappers, fonts, tooltips, shortcuts, keyboard routing, frame callbacks) are grouped under `utils/gui/`, and the non-visual layers may use only the dpg-free helpers that live directly under `utils/` (e.g. `utils/callbacks/`). +Calls into `dearpygui` are confined to `ui/`, `shell.py` and the narrow coordinator surface the Layer Contracts name. The `logic/`, `services/`, `view_model/` and `config/` layers remain DPG-free, so they can be instantiated and tested without a running GUI context. + +The rule covers more than `import dearpygui`. The dpg-bound helpers (`DialogsRenderer`, the `dpg_*` wrappers, fonts, tooltips, shortcuts, keyboard routing, frame callbacks) are grouped under `utils/gui/`. The non-visual layers may use only the dpg-free helpers directly under `utils/`, for example `utils/callbacks/`. ### 3. No UI state in logic -Managers and controllers hold domain state only: file paths, dirty flags, domain objects. Widget visibility, button labels, and progress percentages are UI state, and every UI-ready projection is computed by a view model at the moment it is built. +Managers and controllers hold domain state only: file paths, dirty flags, domain objects. Widget visibility, button labels and progress percentages are UI state, and a view model computes every UI-ready projection at the moment it is built. ### 4. View models are immutable snapshots -A view model captures the exact state needed to render one panel at one moment in time — a Pydantic `frozen=True` model produced by the logic layer and consumed by a panel's `update_view()` method. Derived UI flags (button enabled, sub-panel visible) are `@property` computations on the view model, not stored fields. A frozen dataclass is acceptable where the payload does not suit Pydantic validation (`WaveformData` carries numpy arrays). +A view model captures the exact state needed to render one panel at one moment in time. It is a Pydantic `frozen=True` model produced by the logic layer and consumed by a panel's `update_view()` method. Derived UI flags (button enabled, sub-panel visible) are `@property` computations on the view model and are not stored fields. A frozen dataclass is acceptable where the payload does not suit Pydantic validation (`WaveformData` carries numpy arrays). -This means the UI layer can never be in an inconsistent state: it always reflects the last view model it received, and the view model is self-consistent by construction. +The UI layer therefore can never be in an inconsistent state. It always reflects the last view model it received, and the view model is self-consistent by construction. ### 5. Panels communicate via optional callback hooks -A panel never calls coordinator or logic methods directly. Instead it exposes public optional callback attributes (`on_x: Optional[Callback] = None`) that coordinators set during wiring. The panel fires them through `CallbackMixin.call()`, which notes a hook left unset at debug and yields `None`, so partial wiring during construction reads as the expected condition it is. - -A hook the panel consults for state rather than notifies of an event is read through `CallbackMixin.query()`, which preserves the hook's declared return type and takes the answer to assume while the hook is unset. A widget parameter or branch fed by such a hook therefore receives a value of its expected type at every moment, including the window before wiring completes. +A panel never calls coordinator or logic methods directly. It exposes public optional callback attributes that coordinators set during wiring, and it fires them through `CallbackMixin`. An unset hook is a normal state during construction, so firing it does nothing and reading it yields a default of the expected type. -This decouples widget construction (which happens during `create_panel()`) from the moment wiring takes place (which happens in the coordinator's constructor), and lets panels be instantiated without any coordinator present. +Hooks decouple widget construction (during `create_panel()`) from wiring (in the coordinator's constructor). They also let a panel be instantiated without any coordinator. ### 6. DearPyGui's context belongs to the render thread -The thread that created the DearPyGui context is the only one that may build, configure, or destroy an item, and an item freed from another thread is freed with no Python thread state — a crash rather than a glitch. So work reaching the interface from anywhere else arrives on that thread first, through a crossing named for what it carries: a background result is queued for the render loop to drain, a worker's touch of a widget goes through `on_render_thread`, and a gesture DearPyGui gathered is run at the top of a frame. A crossing that would hold the frames up puts its waiting on a thread of its own, and work that needs a drawn frame names the frame it is picked up on. The four crossings, the helpers that make them, and the hazard each one answers are in [`render-thread.md`](application/render-thread.md). +The thread that created the DearPyGui context is the only one that may build, configure or destroy an item. An item freed from another thread is freed with no Python thread state, which crashes the process. Work that reaches the interface from anywhere else therefore arrives on that thread first, through a crossing named for what it carries: + +- A background result is queued for the render loop to drain. +- A worker's touch of a widget goes through `on_render_thread`. +- A gesture DearPyGui gathered is run at the top of a frame. + +A crossing that would hold the frames up puts its waiting on a thread of its own, and work that needs a drawn frame names the frame it is picked up on. [`render-thread.md`](application/render-thread.md) describes the crossings, the helpers that make them and the hazard each one answers. ### 7. Construction flows from the composition root -`Application.__init__` constructs the application graph — managers, controllers, shared services, coordinators, the shell — and wires their callbacks. A tab coordinator in turn constructs the panels, logic objects, and tab-scoped services it owns. Beyond these two sites, no component constructs another major component: every dependency arrives as a constructor argument, and none is obtained through a global lookup. +`Application.__init__` constructs the application graph (managers, controllers, shared services, coordinators, the shell) and wires their callbacks. A tab coordinator constructs the panels, logic objects and tab-scoped services it owns. No other component constructs a major component. Every dependency arrives as a constructor argument, and none is obtained through a global lookup. -**Where a run keeps its settings arrives the same way.** The application is given a `UserProfile` — the pair of files its configuration and its session state live in — and hands each path to the manager that reads and writes it. The entry point names the user's own profile through `UserProfile.user()`, which leaves one place that knows the shipped locations and lets a run be pointed at a location of its own. +**Where a run keeps its settings arrives the same way.** The application is given a `UserProfile`, the pair of files its configuration and session state live in, and hands each path to the manager that reads and writes it. The entry point names the user's own profile through `UserProfile.user()`. One place therefore knows the shipped locations, and a run can be pointed at a location of its own. ### 8. All display text comes from `LanguageManager` -Every user-visible string is looked up on `LanguageManager` by the key the language file spells — `page.panel.text_type.element` — and resolves at the point of use, so a language change takes effect on the next read. `en.yaml` is a flat map keyed exactly this way, which makes the text system the single source of truth, lets a reader hold a key against the language file by eye, and enables future localization. A lookup states its key in a form the `language-keys` hook can read, so every key the code spells names an entry and every entry the file holds is reached. The grammar, the forms a lookup takes, and where each element enum lives are in [`vocabularies.md`](application/vocabularies.md). Log messages are developer-facing and exempt. +Every user-visible string is looked up on `LanguageManager` by the key the language file spells (`page.panel.text_type.element`), and it resolves at the point of use, so a language change takes effect on the next read. `en.yaml` is a flat map keyed exactly this way. That makes the text system the single source of truth, lets a reader hold a key against the language file by eye, and enables future localization. + +A lookup states its key in a form the `language-keys` hook can read, so every key the code spells names an entry and every entry in the file is reached. [`vocabularies.md`](application/vocabularies.md) has the grammar, the forms a lookup takes, and where each element enum lives. Log messages are developer-facing and exempt. ### 9. `tags/` holds only DPG identifiers -The `tags/` package contains only DPG widget string identifiers: `TAG_*` whole tags, and `SUF_*`/`PRE_*` fragments that compose into them. Dimensions, colors, timings, and display strings live in YAML configuration loaded at startup (`layout/`). Every tag reaches its final spelling through one composer, and a constant's name states the tag it composes, which the `tag-names` hook holds it to. The composer, the `TagName` spelling, and the rules a fragment follows are in [`vocabularies.md`](application/vocabularies.md). +The `tags/` package has only DPG widget string identifiers: `TAG_*` whole tags, and `SUF_*`/`PRE_*` fragments that compose into them. Dimensions, colors, timings and display strings live in YAML configuration loaded at startup (`layout/`). Every tag reaches its final spelling through one composer, and a constant's name says the tag it composes, which the `tag-names` hook checks. [`vocabularies.md`](application/vocabularies.md) describes the composer, the `TagName` spelling and the rules a fragment follows. ### 10. Exclusive operations expose a lifecycle-accurate active state -Some operations are mutually exclusive — typically because they are resource-intensive (background worker pools) and running two at once would exhaust memory or contend for a device. Each such operation exposes an `is_active` signal derived from its **own state machine**, true from the moment the operation is *requested* through to its teardown, including any preparatory phase before the background work begins. A signal that starts at the moment of request is the only one that covers the operation's full span; anything derived downstream (such as whether a worker has actually started) opens a window in which a competing operation can slip in. +Some operations are mutually exclusive, typically because they are resource-intensive (background worker pools) and running two at once would exhaust memory or contend for a device. Each such operation exposes an `is_active` signal derived from its **own state machine**. The signal is true from the moment the operation is *requested* until its teardown, including any preparatory phase before the background work begins. Only a signal that starts at the request covers the operation's full span. A signal derived downstream, such as whether a worker has started, opens a window in which a competing operation can slip in. -The composition root composes the per-operation signals into a single *busy authority* — the one source of truth, consulted in two places: +The composition root composes the per-operation signals into a single *busy authority*, the one source of truth. It is consulted in two places: -- **UI enablement** — panels disable the controls that would start a competing operation. -- **Start-time guards** — each operation's entry point consults the authority and declines to start while another operation is active, so exclusivity holds even when a control is reached outside the normal UI path. +- **UI enablement.** Panels disable the controls that would start a competing operation. +- **Start-time guards.** Each operation's entry point consults the authority and declines to start while another operation is active, so exclusivity holds even when a control is reached outside the normal UI path. -A new exclusive operation joins by contributing its `is_active` to the authority and adding a start-time guard; no per-call-site bookkeeping is needed. The authority stores nothing — it is recomputed from the live operations on demand. +A new exclusive operation joins by contributing its `is_active` to the authority and adding a start-time guard. No per-call-site bookkeeping is needed. The authority stores nothing and is recomputed from the live operations on demand. ### 11. Platform and external-tool differences hide behind a backend Protocol -Where behavior depends on the operating system, the desktop environment, or an external command-line tool, that variation is expressed as a `Protocol` with one implementation per target, chosen by a runtime factory — never as platform branches scattered through the callers. The factory probes availability (`locate_program`) and environment (`System.current()`, `XDG_CURRENT_DESKTOP`) and returns the implementation that fits; callers depend only on the Protocol and read identically on every platform. +Where behavior depends on the operating system, the desktop environment or an external command-line tool, that variation is a `Protocol` with one implementation per target, chosen by a runtime factory. Platform branches are not scattered through the callers. The factory probes availability and environment and returns the implementation that fits. Callers depend only on the Protocol and read identically on every platform. -Each tool's quirks stay sealed inside its own implementation and are named in that class's docstring, where a reader meets them beside the code they explain; the guarantee callers depend on — that a saved file carries one of the offered extensions — is enforced once in the API layer above every backend. `utils/file_dialogs/` applies this to native file dialogs: a `FileDialogBackend` Protocol in `protocol.py`, with desktop-portal, `kdialog`, `zenity`, and `tkinter` implementations under `backends/`, selected by `select_file_dialog_backend()`. `sampletones_tools/calibration/referee/` follows the same shape with its `build_referees()` factory. +Each tool's quirks stay inside its own implementation and are named in that class's docstring. The guarantee callers depend on is enforced once, in the API layer above every backend: a saved file has one of the offered extensions. `utils/file_dialogs/` applies this to native file dialogs, and `sampletones_tools/calibration/referee/` follows the same shape. -Ordering the implementations is part of the factory's job: where several are available, the one that expresses the most wins. A save offering several file types is answered by the portal because it alone reports which type was chosen, so an export names its format in the type selector; a backend answering with a name alone leaves the extension to be read from the name, and the API layer settles it either way. +Ordering the implementations is part of the factory's job: where several are available, the one that expresses the most wins. A save that offers several file types is answered by the desktop portal, for example, because it alone reports which type was chosen. ### 12. One dispatcher owns the keyboard -DearPyGui gives every key handler the same global reach, so priority and consume semantics exist where the application builds them. A single `KeyRouter` (`utils/gui/keyboard/`) owns the one `add_key_press_handler` for the whole application and offers each press to registered **scopes** from highest priority to lowest; the first active scope that claims the press ends the walk. Three priorities order the application — a modal dialog above a sequencer sub-panel above the application shortcuts — and each consumer registers one scope stating when it wants keys and which presses it claims. +DearPyGui gives every key handler the same global reach, so priority and consume semantics exist only where the application builds them. A single `KeyRouter` owns the one `add_key_press_handler` for the whole application. It offers each press to registered **scopes** from the highest priority to the lowest, and the first active scope that claims the press ends the walk. A modal dialog ranks above a sequencer sub-panel, which ranks above the application shortcuts. Each consumer registers one scope that says when it wants keys and which presses it claims. -A binding is declared once and read by everyone who prints or fires it: `ShortcutId` names the action together with the category that answers it, and the scheme under `sampletones_config/keybindings/` decides the combination, so a printed key and the handler behind it stay in step by construction. +A binding is declared once and read by everyone who prints or fires it. `ShortcutId` names the action with the category that answers it, and the scheme under `sampletones_config/keybindings/` decides the combination. A printed key and the handler behind it therefore stay in step by construction. -The router is constructed at the composition root and injected into every consumer (principle 7). The scopes, the focus query, the modal stack, the key vocabulary, and how a scheme is chosen, layered, and edited are in [`keyboard.md`](application/keyboard.md). +The router is constructed at the composition root and injected into every consumer (principle 7). [`keyboard.md`](application/keyboard.md) covers the scopes, the focus query, the modal stack, the key vocabulary, and how a scheme is chosen, layered and edited. ### 13. A color is a token, resolved where it is drawn -A color is written as a palette token and stays one until it reaches DearPyGui. `BaseColor` (`utils/palette/colors/`) carries what was written, and its `rgba` property answers with the palette active at the moment of the read, so whoever holds the color follows a palette swap. Every annotation names `BaseColor`; the read happens where the value is handed to a widget, and what a consumer keeps is the token. What DearPyGui has already taken a copy of is registered with `PaletteBindings` rather than remembered by whoever set it, so a palette change is one switch. The `palette-colors` hook holds all three rules (see Enforcement); the color forms and the switch itself are in [`palette.md`](application/palette.md). +A color is written as a palette token and stays one until it reaches DearPyGui. `BaseColor` carries what was written, and its `rgba` property answers with the palette active at the moment of the read, so whoever holds the color follows a palette swap. Every annotation names `BaseColor`. The read happens where the value is handed to a widget, and a consumer keeps the token. + +DearPyGui copies a color when it receives it. `PaletteBindings` therefore registers what DearPyGui has already taken, and a palette change is one switch. The `palette-colors` hook checks these rules (see Enforcement). [`palette.md`](application/palette.md) describes the color forms and the switch. ### 14. An action is declared once; whoever shows it prints it -An **action** is one `ShortcutId` — the name a key press, a menu item, and a context item all reach one behavior by. Declaring one is a chain of four links: the action and the category that answers it, its keys in every shipped scheme, the one call it makes, and the label the keybindings editor lists it by. The `shortcut-actions` check holds every link (see Enforcement). +An **action** is one `ShortcutId`: the name a key press, a menu item and a context item all reach one behavior by. Declaring one takes the action and the category that answers it, its keys in every shipped scheme, the one call it makes, and the label the keybindings editor lists it by. The `shortcut-actions` check holds every link (see Enforcement). -A menu item is a view of an action: `ShortcutManager.add_menu_item(shortcut_id, ...)` takes both the accelerator and the call from the action and keeps the item under it, so a rebind re-prints the key already on screen. A set of actions several menus show is stated by one builder belonging to whoever owns them, and each door decides where to print it. A menu whose contents follow a selection states them when it is opened. The four links, the kinds of action that state their call differently, and the mechanism behind a restated menu are in [`keyboard.md`](application/keyboard.md). +A menu item is a view of an action. `ShortcutManager.add_menu_item(shortcut_id, ...)` takes both the accelerator and the call from the action and keeps the item under it, so a rebind re-prints the key already on screen. One builder, owned by whoever owns the actions, states a set of actions that several menus show, and each menu decides where to print it. A menu whose contents follow a selection states them when it is opened. [`keyboard.md`](application/keyboard.md) covers the kinds of action that state their call differently and the mechanism behind a restated menu. --- ## Enforcement -Two mechanisms keep the codebase aligned with this document. - -**Import-expressible contracts are enforced by a check.** `sampletones_config/boundaries/rules.yaml` states one rule per layer, mirroring the **Must not import** lists in the Layer Reference; the Layer Reference is the source of truth, and a divergence between it and the configuration is itself a defect. The same domain holds the order the repository's packages import each other in, and the layering inside `sampletones_player`, both declared as layer tables in `docs/development/packages.md`. `sampletones_config/boundaries/` declares what the boundaries are, `sampletones_tools/checks/boundary/` holds how they are read and reported, and `sampletones check import-boundary` (a pre-commit hook, run as `uv run sampletones check import-boundary --all`) runs them over the source and scripts trees. A rule names the prefixes it reaches through the groups `boundaries/general.yaml` declares, so the interface several layers stay clear of is written once and each rule names it. Where a layer may consume another layer's data contract while its implementation stays out of reach (logic and the service result types), the rule names the contracts group that stays in reach. The hook audits the entire source tree on every commit (`--all`), so strengthening a rule surfaces violations in files a commit never touched. That property sets the working idiom for structural refactors: turn the stricter rule on first, and let the failing hook enumerate the remaining work. +**Import-expressible contracts are enforced by a check.** `sampletones_config/boundaries/rules.yaml` declares one rule per layer: the prefixes it may reach and the ones it may not. The file says where each boundary runs, and this document says what the boundary is for. The same domain holds the import order of the repository's packages and the layering inside `sampletones_player`. [`packages.md`](packages.md) says what those boundaries mean. -**The identifier vocabularies, the declarations that complete them, and the shapes a case may not take are enforced the same way.** Further checks, each a `sampletones check ` command under `sampletones_tools/checks/`, run whole-tree as pre-commit hooks: +`sampletones check import-boundary` runs the rules over the source and scripts trees, and the hook audits the entire source tree on every commit. Strengthening a rule therefore surfaces violations in files a commit never touched. That sets the working idiom for structural refactors: turn the stricter rule on first, and let the failing hook list the remaining work. -| Hook | Command | What it holds | -|------|---------|---------------| -| `language-keys` | `check language-keys` | Code and `en.yaml` against each other, in both directions: a literal key names an entry, every entry is reached by some lookup, and a lookup states values the check can read (principle 8) | -| `tag-names` | `check tag-names --all` | A tag constant's name against the tag it composes (principle 9) | -| `unused-tags` | `check unused-tags` | Every `TAG_*`/`SUF_*`/`PRE_*` the `tags/` package declares against the reads of it across `src/`, `tests/`, and `scripts/`, where an import alone stands at no reads | -| `palette-colors` | `check palette-colors` | A color as a token up to the moment it is drawn with: an attribute assigned a resolved `rgba`, a theme color filled outside the palette bindings, and a hex literal in the shipped configuration outside `palettes/` (principle 13) | -| `shortcut-actions` | `check shortcut-actions` | Every action against the links it needs: a combination in every shipped scheme, a name the keybindings editor lists it by, and — for an application-scope action — the call it makes, whether its own binding or a family (principle 14) | -| `rendered-literals` | `check rendered-literals` | A case against the text it renders: an equality holding `str(...)` or an f-string against a written-out string pins whatever the platform or the build decided (`guidelines.md` § Tests) | +**The identifier vocabularies, the declarations that complete them and the shapes a case may take are enforced the same way.** Each is a `sampletones check ` command under `sampletones_tools/checks/`, run over the whole tree as a pre-commit hook. The principle a check holds names it where that principle is stated. The checks are global by nature, because a dead entry and an unread fragment are both absences, so the hooks pass the whole tree and not filenames. -They read the source as an AST through the source layer in `sampletones_tools/checks/source/`, which discovers modules, resolves the receiver a subscript sits on, and expands an enum-annotated key part to its members; the palette and shortcut checks read the shipped YAML beside it. That layer derives each package directory from its own location and reports a root it finds nothing at, so a check that sweeps nothing fails loudly where it would otherwise pass clean. Because the checks are global by nature — a dead entry and an unread fragment are both absences — the hooks pass whole-tree rather than filenames. - -**Behavioral contracts are enforced by review.** Contracts a grep cannot see — where state lives, which methods touch DPG, how errors travel — are upheld in code review against this document. A change that alters one of them lands with the edit stating the new contract, and one that knowingly leaves a distance behind lands with an entry in `docs/development/bugs-and-todos.md § Architecture` — `guidelines.md` § Documents holds that rule. The ledger, not the codebase, is the memory of what is currently out of line. +**Behavioral contracts are enforced by review.** Contracts a grep cannot see, such as where state lives, which methods touch DPG and how errors travel, are upheld in code review against this document. A change that alters one lands with the edit that states the new contract. A change that knowingly leaves a distance behind lands with an entry in [`bugs-and-todos.md`](bugs-and-todos.md#architecture). [`documentation.md`](documentation.md#upkeep) holds that rule. The ledger is the memory of what is currently out of line. --- -## Layer Reference +## Layer Contracts ### `ui/` — View layer -**Purpose:** Constructs and updates the DearPyGui widget tree. Panels own their DPG tags and the widget subtree rooted at `self.tag`. +Constructs and updates the DearPyGui widget tree. A panel owns its DPG tags and the widget subtree rooted at `self.tag`. -**Contracts:** -- A panel creates its entire widget tree in one call to `create_panel(parent)`, rooting its subtree at `self.tag` inside the coordinator-injected `parent`, and calls DPG afterward only in `update_view()`, `update_*` methods, and event callbacks wired by DPG itself. -- Panels hold only visual state: their tag, their child widget references, and layout dimensions. Domain objects stay in logic; panels receive projections of them. -- A panel never encodes its own placement: it does not compose a column tag (`SUF_PANEL_*`) as its parent, and it never hosts a sibling panel. Tab layout is the coordinator's (see the Coordinators reference). Where a section is a card, one card is one panel is one module; the coordinator declares which cards a tab contains and how they are arranged. -- Structural depth themes are bound only by the layout primitives, never by a panel or coordinator. The `TabColumns` scaffold binds each column its declared depth theme — recessed GROUND for a column hosting a stack of floating cards, raised SURFACE for a full-height column that is itself a single docked surface (a file tree, an instrument list) — the `card()` context manager binds SURFACE to a card, and `well()` binds recessed GROUND to a padded region sunk inside one, so a list reads as one body rather than as content loose on its card. Panels and coordinators bind only semantic/content themes (a per-channel checkbox tint, the player toolbar), never GROUND or SURFACE. +- A panel creates its entire widget tree in one call to `create_panel(parent)`, rooting its subtree at `self.tag` inside the coordinator-injected `parent`. Afterward it calls DPG only in `update_view()`, `update_*` methods, and event callbacks wired by DPG itself. +- Panels hold only visual state: their tag, their child widget references and layout dimensions. Domain objects stay in logic, and panels receive projections of them. +- A panel leaves its own placement to the coordinator. It does not compose a column tag (`SUF_PANEL_*`) as its parent, and it does not host a sibling panel. Tab layout belongs to the coordinator. Where a section is a card, one card is one panel is one module, and the coordinator declares which cards a tab contains and how they are arranged. +- Only the layout primitives bind structural depth themes. The `TabColumns` scaffold binds each column its declared depth theme: recessed GROUND for a column hosting a stack of floating cards, and raised SURFACE for a full-height column that is itself a single docked surface, such as a file tree or an instrument list. The `card()` context manager binds SURFACE to a card. `well()` binds recessed GROUND to a padded region sunk inside a card, so a list reads as one body and not as content loose on its card. Panels and coordinators bind the semantic and content themes alone, such as a per-channel checkbox tint or the player toolbar. - Every mutation from outside goes through `update_view(view_model)` or through a direct DPG call (`dpg_configure_item`, `dpg_set_value`) triggered by an `update_*` method. -- Callback wiring from coordinators sets public `on_x` attributes *after* construction; panels must therefore tolerate `None` hooks until wiring is complete. -- Hooks and view models are how a panel reaches state. A widget that queries per-item state *while it draws*, where projecting the whole collection per repaint would be disproportionate, declares one consumer-owned `Protocol` of exactly the queries that draw makes (e.g. `TreeLogicProtocol`, through which the file trees query per-node favorite and playability state); the owning coordinator constructs the real logic object and injects it, and the panel types against the Protocol. One panel holds one such Protocol: a second is the sign that the panel holds two jobs, and the panel divides. -- Dialog presentation belongs to coordinators: a panel fires an intent hook, and the owning coordinator renders the dialog via `DialogsRenderer` with text resolved there. Reusable modal *editing* windows subclass `GUIWindow` and follow the ordinary panel contracts. - -**Sub-structure:** - -| Path | Role | -|------|------| -| `ui/elements/` | Reusable low-level widgets: `GUIPanel` (the panel base class), `GUIWindow` (modal variant), buttons, tables, graphs, trees, fonts, the status bar, and `MenuSection` — a run of menu items restated each time its menu is opened | -| `ui/elements/layout/` | Reusable layout primitives: `TabColumns` (the tab column scaffold), the `card()` context manager and the `well()` inset region, driven declaratively by tab coordinators, and `centered()`, which stands content in the middle of the width it is offered | -| `ui/panels/` | Domain-level composite panels, organized by feature area | -| `ui/themes/` | DPG themes and per-widget style helpers | -| `ui/resources/` | Icons and image resources loaded at startup | -| `ui/menu.py` | `MenuBar` — the application's top menu bar | - -**May import:** `view_model/`, `utils/`, `categories/`, `tags/`, `layout/`, `constants/`, `sampletones_core` types, `sampletones_shared`. -**Must not import:** `coordinators/`, `logic/`, `services/`, `config/`, `application.py`, `shell.py`, `utils/gui/dialogs` (`DialogsRenderer` is coordinator territory). +- Callback wiring from coordinators sets public `on_x` attributes *after* construction, so a panel tolerates unset hooks until wiring completes. +- Hooks and view models are how a panel reaches state. A widget that queries per-item state *while it draws*, where projecting the whole collection per repaint would be disproportionate, declares one consumer-owned `Protocol` of exactly the queries that draw makes. The file trees do this through `TreeLogicProtocol` for per-node favorite and playability state. The owning coordinator constructs the real logic object and injects it, and the panel types against the Protocol. One panel holds one such Protocol. A second one is the sign that the panel does two jobs, and the panel divides. +- Dialog presentation belongs to coordinators: a panel fires an intent hook, and the owning coordinator renders the dialog via `DialogsRenderer` with text resolved there. Reusable modal *editing* windows subclass `GUIWindow`, follow the ordinary panel contracts, and hold to the geometry contract in [dialogs](application/dialogs.md). --- ### `view_model/` — Projection layer -**Purpose:** Bridges the logic layer and the UI layer. A view model is the UI's contract with the logic layer: it specifies exactly what data a panel needs to render itself, pre-computed and immutable. +A view model is the UI's contract with the logic layer: it states exactly what data a panel needs to render itself, pre-computed and immutable. -**Contracts:** -- All view model classes are frozen (see principle 4). Fields are never mutated; a new instance is produced on each logical state change. -- Derived UI flags (`button_enabled`, `panel_visible`, `is_done`) are `@property` computations, not stored fields, to prevent inconsistency. -- View models carry only what is needed for rendering. They must not expose raw domain objects that a panel could mutate. -- Edit payloads — frozen `*Update` models a panel emits through its `on_*_changed` hooks — also live here: they are the UI's outbound contract, the mirror of view models. -- Domain data containers (frozen dataclasses that wrap core types and are used across logic and services) belong in `logic/`. A type belongs in `view_model/` only if its purpose is to carry data across the UI boundary — a panel-feeding snapshot, an edit payload, or a projection a display renders (`WaveformData`). - -**Naming convention:** `ViewModel`, e.g. `ConverterViewModel`, `SequencerTrackerViewModel`. - -**May import:** `constants/`, `sampletones_core` and `sampletones_player` types, `sampletones_shared`, Python standard library. -**Must not import:** `ui/`, `coordinators/`, `logic/`, `services/`, `config/`. +- All view model classes are frozen (see principle 4). A new instance is produced on each logical state change. +- Derived UI flags (`button_enabled`, `panel_visible`, `is_done`) are `@property` computations and not stored fields, so two readings of one state agree by construction. +- A view model carries what a panel renders, and hands out projections a panel can read alone. +- Edit payloads also live here: the frozen `*Update` models a panel emits through its `on_*_changed` hooks. They are the UI's outbound contract, the mirror of view models. +- Domain data containers (frozen dataclasses that wrap core types and are used across logic and services) belong in `logic/`. A type belongs in `view_model/` only if its purpose is to carry data across the UI boundary: a panel-feeding snapshot, an edit payload, or a projection a display renders (`WaveformData`). --- ### `logic/` — Domain layer -**Purpose:** Owns domain state and implements the state-machine transitions that govern it. No knowledge of the UI framework. - -**Key concepts:** +Owns domain state and implements the state-machine transitions that govern it, knowing nothing of the UI framework. -*Managers* own a domain object's lifecycle (load, save, close). They hold the current object, a `Session` that tracks dirty state, and fire `CallbackMixin` callbacks when the state changes. +*Managers* own a domain object's lifecycle (load, save, close). They hold the current object and a `Session` that tracks dirty state, and they fire `CallbackMixin` callbacks when the state changes. -*Controllers* are thin mutation façades over a manager. `ProjectController` exposes named, typed mutation methods (`set_title`, `add_sample`, …) and emits a finer-grained callback per mutation kind (`on_info_changed`, `on_samples_changed`, …). This lets the UI respond precisely to what changed. `ProjectController.batch()` widens that grain to a whole gesture: each mutation still applies the moment it is made, while the callbacks it raises wait for the scope to close and then arrive once each, so a gesture writing hundreds of rows rebuilds its subscribers once. +*Controllers* are thin mutation façades over a manager. `ProjectController` exposes named, typed mutation methods and emits a finer-grained callback per mutation kind, so the UI answers exactly what changed. Its `batch()` widens that grain to a whole gesture. Each mutation still applies the moment it is made, while the callbacks wait for the scope to close and then arrive once each. A gesture that writes hundreds of rows therefore rebuilds its subscribers once. -*Logic objects* (e.g. `ConverterLogic`) orchestrate multi-step workflows within a feature area. They subscribe to services and translate service results into view model updates. +*Logic objects* (for example `ConverterLogic`) orchestrate multi-step workflows within a feature area. They subscribe to services and translate service results into view model updates. -`logic/history/` implements the session-scoped undo engine (`HistoryManager`); its invariants and mechanics are documented in `docs/development/application/undo.md`. - -`logic/reconstruction/browser/` builds the tree of reconstructions both browser tabs render (`BrowserManager`); its pipeline, node vocabulary and shaping rules are documented in `docs/development/application/browser.md`. - -**Contracts:** -- Logic classes produce view models and may therefore import `view_model/`; they import neither `ui/` nor `coordinators/`. -- Logic classes never call DPG. +- Logic classes produce view models and may therefore import `view_model/`. They call no DPG. - Callbacks are declared as optional attributes and invoked via `CallbackMixin.call()`. -- Session objects are simple state machines; they fire `on_state_changed` when they transition, without knowing who listens. -- A logic object that drives a service declares a logic-side `Protocol` of exactly the calls it needs (e.g. `ConversionServiceProtocol`) and receives the real service from its coordinator or the composition root; structural typing keeps the dependency inverted. - -**May import:** `sampletones_core`, `sampletones_player`, `sampletones_shared`, `view_model/`, `utils/`, `categories/`, `layout/`, `config/`, and the service **result contract modules** (`services/result.py`, `services/*/result.py`) so handlers can type the tagged unions they match on. -**Must not import:** `ui/`, `coordinators/`, service implementation modules. +- Session objects are simple state machines. They fire `on_state_changed` when they transition, without knowing who listens. +- A logic object that drives a service declares a logic-side `Protocol` of exactly the calls it needs (for example `ConversionServiceProtocol`) and receives the real service from its coordinator or the composition root. Structural typing keeps the dependency inverted. --- ### `services/` — Async worker layer -**Purpose:** Executes long-running operations (file conversion, waveform regeneration, export, playback synthesis) on background threads and delivers typed results to the main thread via `CallbackQueue`. - -**Contracts:** -- Every service inherits `ServiceBase[ResultType]`, which provides `subscribe(handler)`, `unsubscribe(handler)`, and `_emit(result)`. -- `_emit` always posts the result to `CallbackQueue`; it never calls a handler directly from the background thread. -- Result types are a tagged union of `ServiceStarted`, `ServiceProgress`, `ServiceIntermediate`, `ServiceSuccess`, `ServiceError`, `ServiceCanceled`, enabling exhaustive `match` handling by subscribers. -- A service is one subpackage holding `service.py` and `result.py`, so its implementation and the contract its subscribers type against are reached separately; the generic contracts every service reports through are `services/result.py`. `ServiceProgress.fraction` is the one reading a bar draws, counting the item under way for the part of it that is done — see `docs/development/progress.md`. -- Services hold no references to panels, view models, or logic objects. +Runs long operations (file conversion, waveform regeneration, export, playback synthesis) on background threads and delivers typed results to the render thread. -**May import:** `sampletones_core`, `sampletones_shared`, `utils/callbacks/`. -**Must not import:** `ui/`, `view_model/`, `coordinators/`, `logic/`, `config/`. +- Every service inherits `ServiceBase[ResultType]`, which provides `subscribe(handler)`, `unsubscribe(handler)` and `_emit(result)`. +- `_emit` posts the result to `CallbackQueue`, which puts every handler on the render thread (principle 6). +- Result types are a tagged union of `ServiceStarted`, `ServiceProgress`, `ServiceIntermediate`, `ServiceSuccess`, `ServiceError` and `ServiceCanceled`, so a subscriber matches exhaustively. +- A service is one subpackage holding `service.py` and `result.py`, so its implementation and the contract its subscribers type against are reached separately. The generic contracts every service reports through are in `services/result.py`. `ServiceProgress.fraction` is the one reading a bar draws. See [`progress.md`](progress.md). +- A service knows no panel, view model or logic object. --- ### `coordinators/` — Orchestration layer -**Purpose:** Coordinators are the glue between the UI, logic, and service layers. They own the panels and logic objects for one feature area, wire their callbacks, and handle cross-cutting concerns (dialogs, navigation, session state). - -There are two coordinator kinds: - -*Domain coordinators* manage a cross-cutting concern that spans the whole application lifecycle — e.g. `ProjectCoordinator` (project file I/O, save confirmations), `PlaybackRouter` (the single transport over the shared output device, acting on the active tab's source or the engaged one — see `docs/development/application/playback.md`), or `EditRouter` (the single edit surface behind the menu bar's Edit menu, which shows the actions of the grid holding the cursor — see `docs/development/application/sequencer-blocks.md`). - -*Tab coordinators* own everything for one tab: they instantiate its panels, logic objects, and tab-scoped services, wire their callbacks together, and provide `create_tab()` — the single method that builds the DPG widget tree for that tab. Tab coordinators present a narrow public API of intent-level methods (`set_input_path`, `display_reconstruction`, …) and keep their panels and logic objects private. - -`create_tab()` is the sole authority for the tab's layout: it declares the column and card arrangement through the shared `ui/elements/layout` primitives (`TabColumns`, `card()`) and injects each panel's parent container via `create_panel(parent)`. It builds widgets only — initial view population (pushing the first view models, refreshing trees) runs afterward from the coordinator's post-build initialization, invoked once the whole tree exists, rather than inside `create_tab()`. - -**Contracts:** -- A coordinator touches DPG only on a narrow, closed surface: inside `create_tab()`, and when building dialog content inside a closure passed to `DialogsRenderer.show_modal`. A dialog that must wait for the next frame is deferred through `FrameCallbackManager`. All other presentation goes through `DialogsRenderer`. -- File selection runs through OS-native dialogs, which live outside DPG. A coordinator opens one via `utils/file_dialogs` — a synchronous call that blocks until the user picks a path or cancels — resolves the dialog title and filter name from `LanguageManager`, and routes the returned path through a handler decorated with `@ignore_none_path`, so a canceled dialog is a silent no-op and each handler body runs with a real path. The backend is chosen at runtime; a coordinator never branches on platform. -- A coordinator holds no domain state. It delegates reads and writes to the managers and controllers it was given; what it caches is presentation wiring — resolved language strings, panels, logic objects, callbacks. -- Callbacks received from `Application` as constructor parameters are stored and forwarded as-is. A wrapper is sanctioned where a contract requires an intent-level guard — a busy-authority start-time guard (principle 10) wrapping an operation's entry point — and that guard is the whole of what the wrapper holds. A wrapper that renames a call, reorders its arguments, or adds a step of its own is the coordinator taking on work that belongs to the logic object the call reaches. -- Error dialogs, confirmations, and notices are presented here, with text resolved from `LanguageManager` here (see the Error Handling Policy). -- An export format with choices of its own opens its setup where the save dialog would otherwise ask for the file. The composition root builds one mapping from each such format to its `ExportSetup` (`coordinators/export/setup.py`), and every surface offering an export consults it first, so a surface names no format and a format gains a setup in one place. - -**May import:** `ui/`, `view_model/`, `logic/`, `services/`, `utils/`, `categories/`, `layout/`, `config/`. -**Must not import:** `application.py`, `shell.py`. - ---- - -### `application.py` — Composition root +Coordinators are the glue between the UI, logic and service layers. Each owns the panels and logic objects for one feature area, wires their callbacks, and handles the cross-cutting concerns around them: dialogs, navigation, session state. -**Purpose:** Creates every object in the application and wires all callbacks. Nothing else. +*Domain coordinators* manage a concern that spans the whole application lifecycle. `ProjectCoordinator` handles project file I/O and save confirmations. `PlaybackRouter` is the single transport over the shared output device, acting on the active tab's source or the engaged one (see [`playback.md`](application/playback.md)). `EditRouter` is the single edit surface behind the menu bar's Edit menu, which shows the actions of the grid holding the cursor (see [`sequencer-blocks.md`](application/sequencer-blocks.md)). -`Application.__init__` is the only constructor that may create multiple different coordinator types. After construction it calls `_setup_gui()` to trigger DPG setup and initial view emission, then `run()` starts the event loop. +*Tab coordinators* own everything for one tab. They instantiate its panels, logic objects and tab-scoped services, wire their callbacks together, and provide `create_tab()`. Each presents a narrow public API of intent-level methods (`set_input_path`, `display_reconstruction`, ...) and keeps its panels and logic objects private. -`Application` delegates all domain logic. Its private methods are either event listeners that forward to coordinators or helpers that coordinate two coordinators that cannot reference each other directly. +`create_tab()` is the sole authority for the tab's layout. It declares the column and card arrangement through the shared `ui/elements/layout` primitives and injects each panel's parent container via `create_panel(parent)`. It builds widgets alone. Pushing the first view models and refreshing trees runs afterward, from the coordinator's post-build initialization, once the whole tree exists. -**May import:** everything. +- A coordinator touches DPG on a narrow, closed surface: inside `create_tab()`, and when building dialog content inside a closure passed to `DialogsRenderer.show_modal`. A dialog that must wait for the next frame is deferred through `FrameCallbackManager`. All other presentation goes through `DialogsRenderer`. +- File selection runs through OS-native dialogs, which live outside DPG. A coordinator opens one via `utils/file_dialogs`, a synchronous call that returns once the user picks a path or cancels. It resolves the dialog title and filter name from `LanguageManager`, and routes the returned path through a handler decorated with `@ignore_none_path`. Each handler body then runs with a real path, and a canceled dialog passes quietly. The backend is chosen at runtime, so a coordinator names no platform (principle 11). +- A coordinator holds no domain state. It delegates reads and writes to the managers and controllers it was given. What it caches is presentation wiring: resolved language strings, panels, logic objects and callbacks. +- Callbacks received from `Application` as constructor parameters are stored and forwarded as they stand. A wrapper is sanctioned only where a contract requires an intent-level guard, such as a busy-authority start-time guard (principle 10) around an operation's entry point, and the guard is all the wrapper does. A wrapper that renames a call, reorders its arguments or adds a step of its own is the coordinator taking on work that belongs to the logic object the call reaches. +- Error dialogs, confirmations and notices are presented here, with text resolved from `LanguageManager` here (see the Error Handling Policy). +- An export format with choices of its own opens its setup where the save dialog would otherwise ask for the file. The composition root builds one mapping from each such format to its `ExportSetup` (`coordinators/export/setup.py`), and every surface offering an export consults it first. A surface names no format, and a format gains a setup in one place. --- -### `shell.py` — UI shell +### `application.py` and `shell.py` — The root and the frame -**Purpose:** Manages the DPG context lifecycle, the primary window, the tab bar, the shortcut system, and UI utilities (status bar, FPS timer, audio settings window). It performs no domain operations. +`Application` creates every object in the application and wires all callbacks, and does nothing besides. It is the only constructor that may create several kinds of coordinator (principle 7). It delegates every domain decision: each of its private methods either forwards an event to a coordinator or joins two coordinators that hold no reference to each other. -`ApplicationShell.setup()` creates the DPG context, registers shortcuts, binds the `KeyRouter`'s single global key-press handler, builds the main window (menu bar + tab bar + status bar), and starts the `CallbackQueue` worker thread. Tab coordinators are passed to the shell so it can call their `create_tab()` methods in sequence. - -**Must not import:** `logic/`, `services/`. The shell reaches domain behavior only through the coordinators and callbacks it was handed. +`ApplicationShell` owns the DearPyGui context lifecycle, the primary window, the tab bar, the shortcut system and the utilities around them: status bar, FPS timer, audio settings window. It carries out no domain operation, and it imports neither `logic/` nor `services/`. It reaches domain behavior through the coordinators and callbacks it was handed. --- ### Supporting packages -| Package | Purpose | -|---------|---------| -| `config/` | `ConfigManager` (domain generation config), `SessionManager` (runtime session: last paths, audio device, window geometry). A session file outlives the files and folders it names, so each path it holds is read against the disk where it is used: a dialog opens at the nearest folder still standing, and a file that fails to open is let go of as it fails. Presentation-free: it records load outcomes (`ConfigLoadOutcome`) as domain data for `ConfigCoordinator` to present. Must not import the visual packages, `coordinators/`, or `application.py` | -| `categories/` | `LanguageManager`, the `Page / Panel / TextType / Widget` enum hierarchy, the `AbstractElement` base and the panel element enums under `categories/elements/`, and the key grammar under `categories/key/`. It also holds the message bundles that resolve a whole conversation's words in one place — `export.py`, `exports.py`, `instrument.py`, `pitch.py` — so a coordinator reads its texts once and hands the bundle to whoever phrases the outcome | -| `constants/` | Application-scope facts that carry no behavior, one module per subject — `keybindings.py` names the scheme a build ships, which both the shortcut catalog and the session config read, and `playback.py` names the follow mode, which the session config, the song player, the view models and the menu all state. A fact shared beyond the application belongs to `sampletones_shared/constants/` | -| `layout/` | Pydantic models loaded from YAML at startup; injected into coordinators and panels as `LayoutConfig` | -| `tags/` | DPG widget tags (`TAG_*`), the fragments composing into them (`SUF_*`, `PRE_*`), and `compose_tag` | -| `utils/` | dpg-free helpers usable by any layer (`utils/callbacks/`, color, threading, and `utils/file_dialogs/` — OS-native file dialogs behind a `FileDialogBackend` Protocol, with the D-Bus desktop-portal client under `utils/file_dialogs/backends/portal/`). DPG-bound helpers live in `utils/gui/` and are off-limits to the non-visual layers | -| `viewport.py` | Manages DPG viewport geometry and fullscreen state | - ---- - -## Data Flow Patterns - -### Pattern A: User action → UI update (synchronous) - -User interaction in a panel fires a DPG callback. The panel invokes its own `on_x` hook, which was wired by the coordinator to a logic method. The logic method mutates state and, if needed, produces a new view model and calls `on_view_changed`. The coordinator (or the logic object itself) calls `panel.update_view(new_view_model)`, which drives DPG calls. - -```mermaid -sequenceDiagram - participant UI as GUIConfigPanel - participant CFG as ConfigManager - participant COORD as MainTabCoordinator - participant DPG as DPG - - UI->>CFG: on_audio_settings_changed(update) - CFG->>COORD: config_change_callback() - COORD->>UI: update_view(ConfigPanelViewModel(...)) - UI->>DPG: set_value(tag, value) ×N -``` - -### Pattern B: Background service result → UI update (asynchronous) - -A service running on a background thread emits a result. `ServiceBase._emit()` posts it to `CallbackQueue` with the configured priority. On the next main-thread frame, `CallbackQueue.process()` dispatches it to the logic object's handler. The handler produces a new view model and the panel updates. - -```mermaid -sequenceDiagram - participant BG as Background Thread - participant SVC as ConversionService - participant CQ as CallbackQueue - participant LOGIC as ConverterLogic - participant VM as ConverterViewModel - participant UI as GUIConverterPanel - - BG->>SVC: _on_progress(task_status, progress) - SVC->>CQ: add(listener, ServiceProgress(...), priority) - Note over BG,CQ: thread boundary crossed here - - loop next frame - CQ->>LOGIC: _on_service_result(ServiceProgress(...)) - LOGIC->>VM: build new ConverterViewModel - LOGIC->>UI: update_view(view_model) - UI->>UI: dpg.set_value / dpg.configure_item - end -``` +`config/` holds `ConfigManager` (the domain generation configuration) and `SessionManager` (the runtime session: last paths, audio device, window geometry). A session file outlives the files and folders it names, so each path it holds is read against the disk where it is used. A dialog opens at the nearest folder still standing, and a file that fails to open is let go of as it fails. The package is presentation-free. It records a load outcome as domain data (`ConfigLoadOutcome`) for `ConfigCoordinator` to present. -### Pattern C: Manager session state change → title/menu update +`categories/` holds `LanguageManager` and the vocabulary a lookup is spelled in (principle 8). It also holds the message bundles that resolve a whole conversation's words in one place, so a coordinator reads its texts once and hands the bundle to whoever phrases the outcome. -A manager's session transitions (e.g. reconstruction loaded, project saved) fire `on_state_changed`. The coordinator forwards this to `Application`, which recomputes the title and menu bar view model and pushes updates to the shell. - -```mermaid -sequenceDiagram - participant SES as ReconstructionSession - participant RCOO as ReconstructionCoordinator - participant APP as Application - participant VPORT as ViewportManager - participant MENU as MenuBar - - SES->>RCOO: on_state_changed() - RCOO->>APP: on_session_state_changed() - APP->>VPORT: update_title(...) - APP->>MENU: update(MenuBarViewModel(...)) -``` +`constants/` holds application-scope facts that carry no behavior, one module per subject. A fact shared beyond the application belongs to `sampletones_shared/constants/`. --- @@ -370,68 +245,33 @@ Each layer has a distinct role in the error-handling chain. The rule of thumb is ### Logic and managers — propagate -Logic classes and managers catch an exception only when they can take a concrete recovery action in place (e.g. retrying with a fallback path). I/O errors (`OSError` and subclasses) from file operations propagate directly to the caller. Catching and repackaging an exception without recovery is forbidden by the coding guidelines. +Logic classes and managers catch an exception only when they can take a concrete recovery action in place, for example retrying with a fallback path. I/O errors (`OSError` and subclasses) from file operations propagate directly to the caller. The coding guidelines forbid catching and repackaging an exception without recovery. -When a manager does recover, it records *what happened* as domain data and lets a coordinator present it. For example `ConfigManager` recovers a malformed configuration by loading defaults and appending a `ConfigLoadOutcome` carrying only domain values; `ConfigCoordinator.present_pending_load_outcomes()` later turns each outcome into the matching dialog with text from `LanguageManager`. +When a manager does recover, it records *what happened* as domain data and lets a coordinator present it. For example, `ConfigManager` recovers a malformed configuration by loading defaults and appending a `ConfigLoadOutcome` that carries only domain values. `ConfigCoordinator.present_pending_load_outcomes()` later turns each outcome into the matching dialog with text from `LanguageManager`. ### Services — the only legitimate broad catch -Services run tasks on background threads. If an unhandled exception escapes the worker, the thread dies silently and `CallbackQueue` never delivers the result. For this reason, `ServiceBase` subclasses must catch the exception at the outer boundary of the async task, wrap it in `ServiceError`, and emit it through `CallbackQueue`. This is the **only** place where catching non-specific exception types is permitted, and it must sit in the top-level task wrapper rather than in helper methods. +Services run tasks on background threads. If an unhandled exception escapes the worker, the thread dies silently and `CallbackQueue` never delivers the result. `ServiceBase` subclasses therefore catch the exception at the outer boundary of the async task, wrap it in `ServiceError`, and emit it through `CallbackQueue`. This is the **only** place where catching non-specific exception types is permitted, and it belongs in the top-level task wrapper and not in helper methods. ### Coordinators — the recovery boundary Coordinators own the decision of what to do when an operation fails. They: - Catch **specific exception types** named by the domain or I/O layer (`OSError`, `LoadReconstructionError`, etc.). -- Present failures to the user via `DialogsRenderer` rather than propagating them further. +- Present failures to the user via `DialogsRenderer` and stop the propagation there. - Handle `ServiceError` results from the tagged union returned by async services. -A coordinator must catch precisely: broad catches (`except Exception`, bare `except`) and deferred typing (`# TODO: specify exception type`) are guideline violations. +A coordinator catches precisely. Broad catches (`except Exception`, bare `except`) and deferred typing (`# TODO: specify exception type`) are guideline violations. ### UI layer — errors arrive as data -Panels perform no error handling. All error conditions arrive as data through coordinator-wired callbacks (`on_error: Optional[Callable[[Exception], None]]`), and a panel may display an error state derived from a view model. Dialog presentation likewise belongs to the coordinator: the panel fires an intent hook, the coordinator presents (see the `ui/` contracts). The one catch permitted inside `ui/` is the widget-level input-validation guard — parsing user keystrokes into a value or `None`. Classifying a rendering failure into a typed domain error, and recovering from it, is a coordinator concern: `InstructionsTabCoordinator._render_instruction` catches the concrete plotting failures (`KeyError`, `IndexError`, `ValueError`) and re-raises them as one `LibraryDisplayError`, which its recovery boundary `_on_instruction_loaded` presents. +Panels perform no error handling. All error conditions arrive as data through coordinator-wired callbacks (`on_error: Optional[Callable[[Exception], None]]`), and a panel may display an error state derived from a view model. Dialog presentation likewise belongs to the coordinator: the panel fires an intent hook, and the coordinator presents (see the `ui/` contracts). The one catch permitted inside `ui/` is the widget-level input-validation guard, which parses user keystrokes into a value or `None`. -### Summary - -```mermaid -graph LR - LOGIC["Logic / Manager\n(propagate)"] - SVC["Service\n(wrap → ServiceError\nvia CallbackQueue)"] - COORD["Coordinator\n(catch specific types,\nshow dialog)"] - UI["UI Panel\n(errors as data)"] - - LOGIC -->|exception| COORD - SVC -->|ServiceError| COORD - COORD -->|on_error callback| UI -``` +Classifying a rendering failure into a typed domain error, and recovering from it, is a coordinator concern. The instructions tab coordinator, for example, catches the concrete plotting failures and re-raises them as one `LibraryDisplayError`, which its recovery boundary presents. --- -## File Layout - -``` -sampletones_application/ -├── application.py ← composition root; owns and wires every component -├── shell.py ← DPG context, main window, tabs, shortcuts -├── viewport.py ← viewport geometry + fullscreen -├── paths.py ← single source of truth for filesystem paths -├── ui/ ← elements/ (reusable widgets), panels/ (per feature area), themes/, resources/, menu.py -├── view_model/ ← immutable snapshots + edit payloads; one subpackage per tab, plus shared/ -├── coordinators/ ← one module per coordinator -├── logic/ ← domain state machines; one subpackage per feature area, plus history/ and shared/ -├── services/ ← ServiceBase + one module or subpackage per background worker -├── config/ ← ConfigManager + SessionManager -├── categories/ ← LanguageManager + lookup enums, with the key grammar under key/ -├── constants/ ← application-scope constants, one module per subject -├── layout/ ← LayoutConfig (Pydantic) + YAML loaders -├── tags/ ← TAG_*, SUF_*, PRE_* identifiers and compose_tag only -└── utils/ ← dpg-free helpers; dpg-bound helpers under utils/gui/ -``` - ---- - -## Naming Conventions Summary +## Naming Conventions | Kind | Convention | Example | |------|-----------|---------| @@ -442,10 +282,8 @@ sampletones_application/ | Manager class | `Manager` | `ReconstructionManager` | | Controller class | `Controller` | `ProjectController` | | Service class | `Service` | `ConversionService` | -| DPG widget tag | `TAG_` + the composed tag, upper-cased | `TAG_MAIN_CONFIG_TABLE_CONFIG_ROW` (`main.config.table.config_row`) | -| Tag suffix | `SUF_` | `SUF_PANEL_LEFT` | -| Tag prefix | `PRE_` | `PRE_RECONSTRUCTION_CHANNEL` | -| Text key | `page.panel.text_type.element` | `global.dialog.label.ok` | | Panel callback hook | `on_` attribute | `on_convert_requested` | | Panel state hook | `can_` or `_` attribute | `can_add_to_sequencer`, `replace_in_sequencer_label` | | Logic callback | `on_` attribute | `on_view_changed` | + +The `tag-names` and `language-keys` hooks enforce the spelling of tags and text keys. diff --git a/docs/development/bugs-and-todos.md b/docs/development/bugs-and-todos.md index 2015d2046..643379dbd 100644 --- a/docs/development/bugs-and-todos.md +++ b/docs/development/bugs-and-todos.md @@ -1,16 +1,17 @@ +# Bugs and to-dos + +This is the working ledger of what is still owed: to-dos, deviations from the architecture, and known +bugs. Read it before starting work in an area. Add an entry when a change knowingly leaves something +behind. Each entry says what is owed and why, and the code and the change that closes it hold the rest. + ## To-dos ### Navigation * Interface scale * Tree navigation using keys -* Moving through the converter's list of gathered recordings with the keyboard. The list holds one - row picked out, which is a selection rather than a position: `ConverterState.selected` names it, - a click sets it, and `Del` reaches it through the `SOURCES` key scope - (`ui/panels/main/converter/listing.py`). What is missing is a cursor the arrow keys move, `Home` - and `End`, and a folder opened and closed from the keyboard — the last of which the list answers - for on its own, since which folders stand open is `OpenFolders` in `ui/elements/stems/` rather - than anything the model records. +* Keyboard navigation of the converter's list of gathered recordings: a cursor the arrow keys move, `Home` + and `End`, and folders opened and closed from the keyboard. * Waveform LOD for zooming * Alt for scrolling graphs * Drag and drop @@ -20,33 +21,26 @@ ### Tracker -The first two entries are also what an imported `.fti` reports as left to the file -(section C of `formats/famitracker.md`), so each one closed is a dimension the import -starts carrying. - -* Release points: `NoteValue.RELEASE` stands in the FamiTracker specification while a note-off cuts - the channel. A release segment would need the playback walk, the NSF driver and `NoteOff` to gain - one. -* Arpeggio modes: a sequence's `setting` byte states absolute. Fixed, relative and scheme need an - enum of their own, and scheme needs the item bit-packing FamiTracker gives it. -* The bend in a Bitphase export. `formats/bitphase/envelopes.py` states the three dimensions it - writes; `NesInstrumentRow` already carries `tone_add` and `tone_accumulation`, so the mapping is - confined to that module. -* A transpose or a volume typed in the sample column of a row holding no sample reaches every - channel. The column summarizes the channels its samples cover, and a row covering none falls - back to all four so a value typed there lands somewhere; the reference slot keeps the narrower - reading and stays empty. +The first two entries are the settings an imported `.fti` reports as left behind (see +[FamiTracker export](../formats/famitracker.md#c-reading-an-instrument-file)). Each one closed is a +dimension the import starts carrying. + +* Release points. A note-off cuts the channel today. A release segment needs the playback walk, the NSF + driver and `NoteOff` to gain one. +* Arpeggio modes. A sequence's `setting` byte says absolute. Fixed, relative and scheme need an enum of + their own, and scheme needs the item bit-packing FamiTracker gives it. +* The bend in a Bitphase export. `NesInstrumentRow` already has `tone_add` and `tone_accumulation`, so the + mapping stays inside `formats/bitphase/envelopes.py`. +* A transpose or a volume typed in the sample column of a row with no sample reaches every channel. The + column falls back to all four channels so the value lands somewhere, and the reference slot keeps the + narrower reading and stays empty. ### Workflow * Waveform construction preview for single-file conversion -* Picking several rows of the converter's list at once, so a group leaves or settles in one gesture - rather than a row at a time. The widget family already draws a multi-pick reading — - `StemsListOffer.picking` with `picked_keys` and `picking_room` in `view_model/shared/stems.py` — - built for the mix chooser and switched off for `GATHERED_SOURCES`. What a converter pick needs - beyond it is a pick with no ceiling, since the chooser's is the room a mix has, and the gestures - the list already offers one row reaching every picked row: removal, a channel box, and the - settings card, which names a single row today. +* Picking several rows of the converter's list at once, so a group leaves or settles in one gesture. The + widget family already draws a multi-pick reading, built for the mix chooser. A converter pick needs one + with no ceiling, and every gesture the list offers one row must reach each picked row. * Selection operations on a reconstruction * Reconstruction trimming @@ -54,174 +48,113 @@ starts carrying. * In-application guide/tutorial * Language selector -* Verifying a bend against the criterion. The plan for the refinement carried a guard: render the - bent candidate, score it, and keep the bend only where the cost improves. It was measured and - left out. The criterion agreed with the reading on **every** bent frame of both a matched and a - mismatched target, so the guard rejects nothing; and one extra render-and-score per bent frame - measures around **2.1 s per second of audio**, against a whole conversion's ~1.2 s, so it would - nearly triple a run to change no decision. It is worth revisiting only against material where the - reading is shown to misfire. -* Reading only the bins the refinement asks for. `InstantaneousPitch` transforms every bin the - spectrum covers and then reads five of them per frame, so it computes around twenty times the - work its reading uses. On a CUDA build that vanishes; on a CPU build one transform measures a - tenth or more of a short conversion, and a CI runner has measured it at a third. The kernel is a - matrix of one row per bin, so restricting it to the rows the chosen notes name is a slice — what - needs care is that the union of harmonic bins over a whole stream is wider than any one frame's. -* Keeping what a stopped folder scan found. `_walk` in `logic/main/sources/scan.py` reports - `on_stopped` and returns where the reader presses **Stop**, so the recordings met so far go - nowhere, while `_gather` already answers with them. Handing that list to `answer` instead would - let a reader stop a long walk and keep the count they watched climb. What it needs beside it is - `_gather_read`'s empty branch (`coordinators/tabs/main.py`) reworked: a stop before the first - recording turns up is a different answer from a folder that holds none, which is what that branch - says today. -* Calibrating the pitch refinement. `generation.refinement`'s confidence threshold, change weight - and window are chosen by hand; `docs/tools/calibration.md`'s experiment measures the criterion - blend and could measure these beside it. The change weight is the one with an audible trade-off: - it decides how large a one-frame excursion the walk follows rather than absorbs, which is - vibrato against jitter. +* Reading only the bins the refinement asks for. The pitch reading transforms every bin the spectrum + covers and uses a handful of them per frame. On a CPU build one transform is a tenth or more of a short + conversion. Restricting the kernel to the rows the chosen notes name is a slice. The care is that the + union of harmonic bins over a whole stream is wider than any one frame's. +* Keeping what a stopped folder scan found. Stopping a long walk drops the recordings met so far, while the + gathering already answers with them. Handing that list on would let a reader stop a long walk and keep + the count they watched climb. It needs the empty branch of the read reworked beside it, since a stop + before the first recording is a different answer from a folder with none. +* Calibrating the pitch refinement. The refinement's confidence threshold, change weight and window are + chosen by hand, and [the calibration](../tools/calibration.md) could measure them beside the criterion + blend. The change weight is the one with an audible trade-off: it decides how large a one-frame + excursion the walk follows and how large it absorbs, which is vibrato against jitter. ### Technical -* Leading the calibration report with `mr-loudness-dB`. `build_referees` puts `mr-auditory-dB` - first, which reads silence as closer to a tone than any render, until by-ear ratings of a sweep - hold the loudness-weighted referee at ρ ≥ 0.6 in every category. A listening round in September - 2026 scored `polyphony-chord` 6–7 dB better on a render that had dropped the noise channel - entirely, so the bar in `docs/tools/calibration.md` stands unmet and the order stays as it is. -* An axiom stating that a recording built with noise reconstructs with the noise channel sounding. - The corpus knows which items were synthesized from noise and the render records already hold the - per-channel timelines, so such a test would fence the criterion against noise deafness the way - `referee/test_axioms.py` fences the referees, without a listening round. +* Leading the calibration report with `mr-loudness-dB`. The report lists `mr-auditory-dB` first, which + reads silence as closer to a tone than any render. The order changes once by-ear ratings of a sweep hold + the loudness-weighted referee at ρ ≥ 0.6 in every category. A listening round scored `polyphony-chord` + 6–7 dB better on a render that dropped the noise channel entirely, so the bar is unmet. +* An axiom that a recording built with noise reconstructs with the noise channel sounding. The corpus + knows which items were synthesized from noise, and the render records hold the per-channel timelines. + Such a test would fence the criterion against noise deafness without a listening round, as + `referee/test_axioms.py` does for the referees. * API documentation * Code documentation -* What a build makes of the configuration and the session state an older one left behind. Both - carry no version at all, so neither travels a chain and neither is archived beside the stored - formats, whose corpus is now `tests/data/compatibility`. A `state.yaml` naming a panel that has - since gone, or a configuration missing a setting added since, is read by whatever each loader - happens to do with it, which nothing states. -* The element enums that outlived their keys. A lookup states its key literally, so an element - enum is named only where a `_label(element)` helper takes one — `ui/menu.py`, - `coordinators/project.py`, `coordinators/keybindings.py`, `ui/panels/dialogs/project_properties.py` - and the panels beside them. The language-keys check expands such a helper over the whole enum, so - a member no call names is reached all the same and stands unnoticed. Spelling those keys literally - at the call site would make each entry exactly checkable and retire the enums that remain. -* Respecting FamiTracker limitations at the writers. A target format's ceilings belong to the - code that writes that format: an envelope carries whatever length a reader wrote, and meets a - limit where a file is built. `formats/famitracker/sequences/features.py` is where the 252-item - sequence ceiling applies today, and it is the one place that decides what a file holds, which - both the export's report and the instruments panel's warning read. What is still owed is the - same treatment for the ceilings a module carries — the instrument, sequence and pattern counts - in `specification/` — so a project past one of them is reported to the reader rather than - refused by the writer. +* What a build makes of the configuration and the session state an older one left behind. Neither has a + version, so neither travels an upgrade chain or has an archived corpus. A `state.yaml` naming a panel + that has since gone, or a configuration missing a setting added since, is read by whatever each loader + happens to do with it. +* The element enums that outlived their keys. An element enum is named only where a `_label(element)` + helper takes one, and the language-keys check expands such a helper over the whole enum, so a member no + call names is reached all the same. Spelling those keys literally at the call site makes each entry + exactly checkable and retires the enums that remain. +* Respecting FamiTracker limits at the writers. A target format's ceilings belong to the code that writes + it, and today only the sequence-length ceiling follows that rule. The instrument, sequence and pattern + counts should follow it too, so a project past one of them is reported to the reader and not refused by + the writer. * Per-tab undo routing +* A history of its own for a standalone reconstruction document, one loaded from disk and not opened as a + project sample. The engine is session-scoped to a project, so an edit to such a document is undoable + nowhere. Giving it a stack reuses the same engine ([undo](application/undo.md)). * In-application console -* Improve performance of browser favorite scan of the entire tree per click +* Improve performance of the browser's favorite scan of the entire tree per click ## Architecture -Where the codebase stands apart from `docs/development/architecture.md`. A deviation is recorded -here once review has seen it and let it stand, so this section — rather than the code — is the -memory of what is currently out of line, and an entry leaves when the code meets the contract -again. - -* The two sequencer grids state the same machinery twice. `ui/panels/sequencer/order/` and - `ui/panels/sequencer/tracker/` each divide into a panel and its collaborators, and the panel - modules still declare the same block and channel hooks, build the same `ChannelSwitch`, tint a - channel the same way, and run the same held-pointer drag, right-click hit-test and key dispatch. - `ui/panels/sequencer/grid/` is where both already reach for what they share, and it is where - these belong; the `# TODO: to abstract` markers stand at the blocks themselves. Pylint's - duplicate-code report names each pair, so the work is enumerable rather than a matter of - reading. -* `application.py` and `ui/elements/tree/tree.py` each hold several concerns in one module, past - the size at which the sequencer panels and the sequencer tab coordinator were divided into - subpackages. Each divides the same way: a module per concern, with the class that stays holding - the collaborators and the public surface. -* The Main tab's public surface renames several calls on its way to a panel — - `refresh_converter_view`, `is_converter_panel_visible`, `refresh_browser`. These are the tab's - own face rather than the inbound callbacks the Coordinators contract governs, which now travel as - one `MainTabHooks` value and are forwarded as they stand. What is worth settling is whether the - face wants those names at all, or whether the application should ask for the thing rather than - for the refresh of it. -* `utils/gui/dpg.py::dpg_get_item_parent` catches `Exception` where the Error Handling Policy leaves - the broad catch to a service's top-level task wrapper. It stays: DearPyGui raises `Exception` - itself for an absent item rather than a type of its own, so the catch is as narrow as what it - answers. A test pins that contract, and the day the library raises something of its own is the - day the catch narrows. -* Every fixture in `tests/unit/sampletones_application/coordinators/tabs/test_main.py` builds - `MainTabCoordinator` through `__new__` and populates its privates by hand, so what those cases - describe is a method rather than the wired object. The wiring itself is exercised — - `test_startup.py` builds the real application and drives gestures through it end to end — so the - gap is that a case reading the coordinator's own behavior cannot see a hook left unset. Building - the object in that file is what closes it. -* Principle 6 was rewritten once on the premise that a widget's callback arrives on the render - thread, reasoned from `manual_callback_management` never having been enabled. A probe reads the - opposite: a global mouse handler reports one thread identifier and the render loop another, so - DearPyGui answers a gesture on a thread of its own and every callback that rebuilt widgets was - racing the renderer. Manual callback management is now on and the frame runs what DearPyGui - gathered, which makes the principle true rather than merely stated. What a gesture costs is now - paid between frames, so a callback heavy enough to be felt is one to spread across frames itself. -* `state.last_paths.library` is written and never read. `SessionManager.set_library_path` records - the directory a library was chosen from, and `get_library_path` is reached by no caller: the - dialog that would open there takes its starting directory from the advanced settings panel - instead. Either the dialog reads the remembered path or the field and its pair of accessors go. -* Which recordings a mix is built from is decided in `ui/` until **Add** is pressed. The chooser - holds the pick as its own `_picked` set and asks `StemsListViewModel` how a gesture moves it — - `picking_of`, `reaches` and `picking_settled` compute the transitions in `view_model/shared/`, - which is a projection answering a question about state rather than describing one. The logic - layer hears the answer and nothing before it, so a pick abandoned by closing the window was - never state anyone else could read. Principles 3 and 4 put that machine in `logic/`, with the - chooser drawing what a view model says and reporting the gesture; moving it is a phase rather - than a patch, because the dialog is what drives the pick today. -* `ConverterMessages` reads the strings it puts to a reader once, at construction, where principle - 8 has text resolve at the point of use so a language change takes effect on the next read. The - stage names and the status lines are cached as fields; the templates the run fills are read live. - This predates the converter's rebuild — the class it replaced cached the same way — and the fix - is the same either way: read each key where it is used, and let the manager answer. -* `FolderScan` (`logic/main/sources/scan.py`) runs a long directory read on a worker and reports - back, which is what `services/` is for, while standing in `logic/`. It reports through optional - hooks rather than the result union, and its reports arrive on the worker's own thread, so the - coordinator crosses to the render thread on its behalf rather than the walk posting to - `CallbackQueue`. It stays there because it is short and the tab is its only caller; what a move - would buy is the exhaustive `match` every other long operation reports through. -* Every gesture re-derives the whole setup. `ConverterLogic._settle` reads the gathered sources - into rows and follows the state to its destination, which builds one batch entry per recording - still holding a channel. `tests/benchmarks/test_converter_load.py` holds both to the length of - the list, and reads about 40 ms and 50 ms on a folder of ten thousand with the collector held - off. What a reader pays is more: a gesture hands `_settle` a state whose recordings are new - objects, so the readings are taken cold and the collector's own share falls inside them — - measured together at roughly a quarter of a second per gesture at that size, before a widget is - touched. All of it is repeated work, since what changed was one recording. Answering it means - holding the rows against the gathering that produced them and deriving entries for the - recordings a gesture actually moved. -* Several directories under `ui/` carry modules without an `__init__.py`, which leaves each one a - namespace package. A tool reading the tree treats such a directory as a root it can import from, - so a module inside one answers for a standard-library name of the same word: `ui/elements/trace.py` - stands against `trace` this way, and `ui/elements/graphs/layers/array.py` did against `array` - until it was given a package of its own. Giving each directory an `__init__.py` closes the - whole class. +Where the codebase stands apart from [`architecture.md`](architecture.md). A deviation is recorded here +once review has seen it and let it stand, so this section, and not the code, is the memory of what is +currently out of line. An entry leaves when the code meets the contract again. + +* The two sequencer grids implement the same machinery twice. The order and tracker panels each declare the + same block and channel hooks, build the same `ChannelSwitch`, tint a channel the same way, and run the + same held-pointer drag, right-click hit-test and key dispatch. The shared `ui/panels/sequencer/grid/` + package is where these belong, and `# TODO: to abstract` markers stand at the blocks. Pylint's + duplicate-code report names each pair. +* `application.py` and `ui/elements/tree/tree.py` each hold several concerns in one module, past the size + at which the sequencer panels were divided into subpackages. Each divides the same way: a module per + concern, with the class that stays holding the collaborators and the public surface. +* The Main tab's public surface renames several calls on its way to a panel (`refresh_converter_view`, + `is_converter_panel_visible`, `refresh_browser`). Those are the tab's own face, and the inbound callbacks + the Coordinators contract governs are forwarded as they stand. The open question is whether the face + needs those names at all, or whether the application should ask for the thing and not for the refresh of + it. +* `dpg_get_item_parent` catches `Exception`, where the Error Handling Policy leaves the broad catch to a + service's top-level task wrapper. Its docstring says why, and the catch narrows the day DearPyGui raises + a type of its own. +* Every fixture in the Main tab coordinator's tests builds the coordinator through `__new__` and fills its + privates by hand, so those cases describe a method and not the wired object. A case reading the + coordinator's own behavior cannot see a hook left unset. Building the object in that file closes the + gap. +* `state.last_paths.library` is written and never read. `SessionManager.set_library_path` records the + directory a library was chosen from, and no caller reaches `get_library_path`. Either the dialog reads + the remembered path or the field and its accessors go. +* Which recordings a mix is built from is decided in `ui/` until **Add** is pressed. The chooser holds the + pick as its own set and asks the view model how a gesture moves it, so a projection computes state + transitions. The logic layer hears only the answer. Principles 3 and 4 put that machine in `logic/`, + with the chooser drawing what a view model says and reporting the gesture. Moving it is a phase and not + a patch, because the dialog drives the pick today. +* `ConverterMessages` reads the strings it shows a reader once, at construction, where principle 8 has text + resolve at the point of use. The stage names and status lines are cached as fields, and the run's + templates are read live. The fix is to read each key where it is used and let the manager answer. +* `FolderScan` runs a long directory read on a worker and reports back, which is work that `services/` exists for, + while it stands in `logic/`. It reports through optional hooks and not the result union, and the + coordinator crosses to the render thread on its behalf. Moving it would buy the exhaustive `match` every + other long operation reports through. +* Every gesture in the converter re-derives the whole setup. A gesture hands `ConverterLogic._settle` a + state whose recordings are new objects, so the readings are taken cold and the garbage collector's own + share falls inside them. On a very large folder that is long enough to feel as a pause before a widget + is touched, and all of it repeats work, since one recording changed. The answer is to hold the rows + against the gathering that produced them and derive entries for the recordings a gesture moved. +* Several directories under `ui/` have modules without an `__init__.py`, which makes each a namespace + package. A tool reading the tree treats such a directory as a root it can import from, so a module + inside one answers for a standard-library name of the same word (`ui/elements/trace.py` against + `trace`). Giving each directory an `__init__.py` closes the whole class. ## Bugs -* Misaligned dialog boxes sizes at initialization -* The instruments panel draws the part a reader hears, so a frame held by a recording left out - reads as a rest rather than naming its owner. A DearPyGui bar series takes one color for the - whole series, so naming it means a series per owner or a drawn overlay beneath the plot. -* `ReconstructionManager` writes `_reconstruction_hash` and `_coefficient` and reads neither. -* A channel whose every frame rests reads as standing by for the reader while `playing_channels` - and the export still count it, so the panel and the size figures disagree about a channel whose - volume was written down to nothing. -* `ETAEstimator` states the mark for an estimate not yet measurable as a literal in - `sampletones_core`, where the language file the rest of the application reads its words from - is out of reach. Lifting the phrasing to the layer that shows it is what settles it. -* `_on_bar_point_clicked` composes a raw-data tag the text field does not carry, so its write - finds nothing and the field catches up only when the edit returns through the regeneration. -* A reconstruction written before the recorded sources moved onto the stems record reads with - none of them, so the browser, original playback and the Stems card see a detached document - where the file names its recordings under the top-level `audio_filepath`. Folding those paths - onto `StemsData.sources` is what the reconstruction 2.2 conversion step still owes, along with - the per-entry drives and channel count a mid-branch 2.2 file carries at the top of its setup. -* `ReconstructionStage.RENDERING` keeps the weight it was measured at while a conversion no longer - renders, so a bar covers that share faster than the eight parts in a hundred `STAGE_WEIGHTS` - gives it. Re-measuring the four stages over whole runs is what settles the new figures. -* `performance/audition.py` previews an instrument at unit drive, where a reconstruction's own - frames sound at the drive their recording gives the channel. Which level a preview belongs at is - worth stating either way. +* A channel whose every frame rests reads as standing by for the reader, while `playing_channels` and the + export still count it. The panel and the size figures therefore disagree about a channel whose volume was + written down to nothing. +* `_on_bar_point_clicked` composes a raw-data tag the text field does not have, so its write finds nothing + and the field catches up only when the edit returns through the regeneration. +* A reconstruction written before the recorded sources moved onto the stems record reads with none of them. + The browser, original playback and the Stems card then see a detached document, where the file names its + recordings under the top-level `audio_filepath`. The 2.2 conversion step still owes folding those paths + onto `StemsData.sources`, along with the per-entry drives and channel count a mid-branch 2.2 file has at + the top of its setup. +* `ReconstructionStage.RENDERING` keeps the weight it was measured at, while a conversion no longer + renders. A bar therefore covers that share faster than `STAGE_WEIGHTS` says. Re-measuring the stages over + whole runs settles the new figures. diff --git a/docs/development/documentation.md b/docs/development/documentation.md new file mode 100644 index 000000000..a2783681a --- /dev/null +++ b/docs/development/documentation.md @@ -0,0 +1,126 @@ +# Writing the documentation + +This document governs the prose in this repository: the README, the guide, the development documents, the +reference pages, and the changelog. Consult it before adding a page, and before editing one. The +conventions for code and docstrings are in [guidelines.md](guidelines.md). + +## Every document has one reader + +Each document serves one reader, and that reader decides the rest: what belongs on the page, how much of +it, and the words it is written in. A passage that serves a different reader belongs in that reader's +document, or nowhere. + +| Document | Reader | What they came for | +|---|---|---| +| `README.md` | someone who just found the project | what it is, and how to run it | +| `docs/guide/` | a beginner using the application | how to do the thing they want to do | +| `docs/tools/` | someone running a measurement command | what the command does, and how to run it | +| `docs/development/`, `docs/concepts/` | a developer or agent about to change the code | the design, and the reasons behind it | +| `docs/formats/`, `docs/api/` | a programmer reading or writing our files | the exact shape of a format or a call | +| `CHANGELOG.md` | the people who use each release | what changed for them | + +## README + +Say what the project is, what it needs, how to install it, and how to run it. Keep it as plain as +possible, and no plainer. A reader decides here whether to try the application at all, so nothing else +competes for their attention. Detail belongs in the guide, and mechanism in the development documents. + +## The guide + +Write for a beginner who wants to get something done, and assume no prior knowledge. Help the reader +act: name the control they click, in the order they would reach it. Keep each topic short — a few +sentences per feature — and leave out how the application works inside. + +Use plain, direct English. Short sentences, one fact each. Everyday verbs, not this repository's own +vocabulary. A term the reader would not know is either avoided or defined in +[the glossary](../glossary.md) and linked from the page that uses it. + +Two passages read the way a guide page should. "Voices: samples and instruments" in +[the sequencer guide](../guide/sequencer.md) names each kind in one sentence, then says what the reader +does with it. The three install options at the top of [installation](../guide/installation.md) give each +option a name and one sentence saying who it suits. + +## Tools pages + +One page per command, written for someone about to run it: what the command measures or produces, how to +run it with no options, what it writes, and every custom use. How it works comes last. The guide's +writing rules hold here — plain English, short sentences, a list wherever the page states several things +of one kind. [Tooling](tooling.md) governs what a tool is, and [docs/index.md](../index.md) lists the +pages. + +The opening of [the calibration page](../tools/calibration.md) does this: what it measures, what to use +it for, then how to run it. + +## Development documents + +Write for a developer or an agent about to change the code. This is the reader of +`docs/development/` and of `docs/concepts/` alike. Open by saying what the document governs and when to +consult it, so a reader learns in one paragraph whether they are in the right place. + +What belongs here is what the code cannot say: concepts and definitions, principles, design decisions, +the contracts a layer honors, and the reasons behind them. State what a mechanism achieves, generally. + +**Apply one test to every paragraph: if reading the code would tell you the same thing, it goes.** A walk +through classes and methods, a call order, a source tree, a table of which file holds what — the code +already says all of that, and says it correctly after the next refactor. So does a copy of a keybinding, a +default or a measured weight: name the constant or the file that holds the value, not the value. + +Lead with principles, then mechanics. Keep the three kinds distinct: a principle is a reason, a convention +is a mechanic that serves it, and a description is a fact about how something works. Prefer a few strong +principles to many narrow rules; a growing list of small rules usually means a principle has gone +unstated. + +Write for a reader who never saw the history. A development document is not a devlog: leave out past +states, resolved problems and rejected alternatives. The design as it stands carries its own +justification. Work still owed goes to [the ledger](bugs-and-todos.md), one brief entry each. + +Two documents to read for the register: [the render thread](application/render-thread.md), where every +section is a decision with the hazard it answers and the mechanism stays general, and +[colors and palettes](application/palette.md), which does the same in forty lines. The design principles +in [architecture.md](architecture.md) show the other half of it: each principle is a claim with its +reason, binding code it never names. + +## Reference pages + +Formats and the Python API are consulted rather than read. Be exact and complete about the shape — the +fields, the order, the units — and let tables carry it. A reader arrives knowing what they are looking +for, so the page needs no narrative. + +## The changelog + +`CHANGELOG.md` belongs to the maintainer, who writes it for the people using each release. Leave it +alone. + +## What holds everywhere + +- A page opens by saying who it is for and when to use it. A reader learns in one paragraph whether they + are in the right place. +- A term is defined or linked before its first use. A reader never meets a word the page has not + introduced. +- One fact per sentence, and short sentences. Several clauses joined by semicolons hide the facts inside + them. +- State each fact once, in the document that owns it, and link the sibling document rather than repeating + it. +- A measurement is written so a reader can place it. State it as a comparison — this against that, taken + together — or give the conditions it was taken under: the material, the machine, the build. A bare + figure carries no meaning a reader can check, so it is dropped or turned into the comparison it stands + for. +- A recipe says what it is for before what to type, and each step that could be done another way says + what it buys. Commands alone leave a reader following instructions they cannot check or adapt. +- State things in positive terms: what the design does, not what it avoids or once did. Reach for a + negative only where the contrast teaches something a positive sentence cannot. +- Use American English. A name someone else owns keeps their spelling: `MatchRule.serialise()` is + jeepney's. + +## Upkeep + +A document is part of the change that motivates it. Code that alters a contract a document states lands +together with the edit stating the new contract, and a deviation the change knowingly leaves behind lands +with an entry in [the ledger](bugs-and-todos.md). + +A development document sits beside what it governs: the top of `docs/development/` holds what spans the +repository's packages, `application/` what governs the graphical application, and `release/` what a +release ships and keeps compatible. [docs/index.md](../index.md) lists every document. + +Deleting is an edit like any other. A page whose reader has gone, or whose content the code now carries, +goes with the change that made it so. diff --git a/docs/development/guidelines.md b/docs/development/guidelines.md index 4e522505f..852cb5ef4 100644 --- a/docs/development/guidelines.md +++ b/docs/development/guidelines.md @@ -1,8 +1,9 @@ # Coding Guidelines These rules govern the Python in this repository. They complement -`docs/development/architecture.md` (ownership and layering) and -`docs/development/application/config-organization.md` (configuration). +`docs/development/architecture.md` (ownership and layering), +`docs/development/application/config-organization.md` (configuration) and +`docs/development/documentation.md` (the prose every document is written in). ## General @@ -10,16 +11,17 @@ These rules govern the Python in this repository. They complement 1. Split a function with several meaningful steps into helpers, each with one responsibility. 1. Spell names out in full: `note`, not `n`. 1. Give every semantic value a name — a `Final` constant, promoted to a shared module once the concept is reused. -1. Avoid the _tramp data_ antipattern: threading a value through functions that only pass it along. +1. Hand a value to the function that uses it. Threading it through functions that only pass it along is the _tramp data_ antipattern. 1. Make the inputs logic depends on explicit. The parameters and configuration instances it relies on are required, not optional. Reserve default values for settings seldom changed (e.g. `seed`), and declare each such default as a top-level `Final` constant. -1. Derive booleans rather than storing them. A boolean computed from existing state belongs in a `@property` (or `@computed_field` on a Pydantic model), since a stored flag creates hidden state that drifts out of sync. -1. State type expectations explicitly, and reach attributes by direct access rather than dynamic `getattr` or `hasattr`. +1. Derive booleans from state. A boolean computed from existing state belongs in a `@property` (or `@computed_field` on a Pydantic model), since a stored flag creates hidden state that drifts out of sync. +1. State type expectations explicitly, and reach attributes by direct access, not through dynamic `getattr` or `hasattr`. 1. Prefer protocols over inheritance. 1. Prefer `match` statements over long `isinstance` chains, and for enumeration handling. 1. Prefer `pathlib.Path` over `os.path`. 1. Separate function options with `*`, and choose positional arguments intentionally. -1. Change internal APIs, configs, and data shapes freely; preserve backward compatibility only when the user explicitly asks. -1. Move a stored data version once per release. The version a build writes between releases is still being written, so a further change to that format extends the upgrade step already pending — one step carries the whole distance from the version the last release shipped. Libraries are rebuilt from their settings, so a change to what library generation produces moves the library version alone. See [data compatibility](release/compatibility.md). +1. Change internal APIs, configs, and data shapes freely. Preserve backward compatibility only when the user explicitly asks. +1. Move a stored data version once per release. The version a build writes between releases is still being written. A further change to that format therefore extends the upgrade step already pending, and one step carries the whole distance from the version the last release shipped. Libraries are rebuilt from their settings, so a change to what library generation produces moves the library version alone. See [data compatibility](release/compatibility.md). +1. A risk named while planning lands as a case or a ledger entry. A plan records intent, and a case and the ledger carry a doubt past the moment it was felt. Where the risk is a behavior that might be wrong, write the case that would catch it. Where it is a distance the change accepts, write the entry in [bugs and todos](bugs-and-todos.md) that names it. 1. Run `pre-commit` on new files after each change. ## Ownership @@ -28,7 +30,7 @@ These rules govern the Python in this repository. They complement 1. Put general-purpose, non-model-specific helpers in shared or common modules. 1. Search the repository with `rg` for existing logic before adding a helper. 1. When new code would duplicate existing logic, extract the shared rule first and route both call sites through it. -1. Import a shared helper straight from the module that implements it; a re-export or delegated-import module that exists only to route imports through is disallowed. +1. Import a shared helper straight from the module that implements it. A re-export or delegated-import module that exists only to route imports through is disallowed. 1. An `__init__` exposes only names from within its own tree hierarchy. 1. Give each module a single area of responsibility. 1. If a module contains many class and function definitions, split into a subpackage divided by a single concern. @@ -51,7 +53,7 @@ These rules govern the Python in this repository. They complement 1. Let a failure crash unless the code can recover from it meaningfully. 1. Handle errors at the execution boundary where possible. -1. Catch an exception only to recover from it; a `try`/`except` that repackages a failure without recovering adds nothing. +1. Catch an exception only to recover from it. A `try`/`except` that repackages a failure without recovering adds nothing. 1. Bare `except` and `except Exception` are forbidden. 1. Scope each `try` to the statements that can actually fail, absent a specific reason to widen it. @@ -64,54 +66,31 @@ These rules govern the Python in this repository. They complement ## Docstrings and Comments 1. A docstring explains the intention of a class or function and the context of its use. -1. State functionality in positive terms. Describe what a class or function *does* — not what it avoids, omits, skips, differs from, or no longer does. Reframe every negation ("does not", "rather than", "instead of", "without", "never", "cannot", "no longer") into the behavior that actually happens. Do not contrast with rejected alternatives as justification; the positive statement carries the meaning. +1. State functionality in positive terms. Describe what a class or function *does*. Reframe every negation ("does not", "rather than", "instead of", "without", "never", "cannot", "no longer") into the behavior that actually happens. The positive statement carries the meaning, so a contrast with a rejected alternative adds nothing. 1. Negative phrasing is allowed only where the condition itself is the contract: exception triggers in `Raises:` clauses, precondition/postcondition bounds (prefer "must be at least X" over "cannot be less than X" where natural), and documented edge-case returns. Outside these concrete cases, negative descriptions are information noise and must be removed. -1. Justify an arbitrary choice in the docstring rather than a code comment, and frame the justification by what the choice achieves. +1. Justify an arbitrary choice in the docstring, not in a code comment, and frame the justification by what the choice achieves. 1. Let clear names carry the meaning, and skip comments or docstrings that restate the code. -1. Avoid code comments; they are warranted for tensor shapes, third-party API quirks, or non-obvious invariants. +1. Avoid code comments. They are warranted for tensor shapes, third-party API quirks, or non-obvious invariants. 1. Code comments and docstrings are not for recording changes or progress. 1. Don't write module docstrings. -## Documents +## Documentation -1. A document under `docs/` explains a subsystem to someone about to change it. Open by stating what it governs and when to consult it, so a reader learns in one paragraph whether they are in the right place. -1. A development document sits beside what it governs. The top of `docs/development/` holds what spans the repository's packages; `application/` holds what governs the graphical application alone, and `release/` what a release ships and keeps compatible. `docs/index.md` lists every document under the heading of its directory. -1. Lead with principles, then mechanics. A principle is a design truth you reason from; state the principles first, and let concrete conventions and reference tables follow as the way each principle is realized. -1. Keep principles, conventions, and descriptions distinct. A principle is a reason; a convention is a handy mechanic that serves it; a description is a fact about how something works. A convention promoted to a principle, or a principle buried in a description, misleads the reader about what is load-bearing. -1. Prefer a few strong principles to many narrow rules. When several rules are facets of one idea, state the idea once and derive them. A growing list of ad-hoc rules signals a principle that has gone unstated. -1. State the design in positive terms, as it stands today. This is the docstring rule above applied to prose: describe what the design is and does, not what it avoids, omits, or once was. -1. Write for a reader who never saw the history. A document is not a changelog or a devlog: do not argue against past states, resolved problems, or rejected alternatives the reader never knew existed. The design as it stands carries its own justification; history belongs in commit messages and release notes. -1. Reach for a negative example only when the contrast teaches something the positive statement cannot, and use it sparingly. One well-placed "what to avoid" illuminates; a document written mostly in negatives is noise. -1. State each fact once, in the document that owns it, and cross-reference sibling documents rather than repeating them. -1. A document change is part of the change that motivates it. Code that alters a contract a document states lands together with the edit stating the new contract, and a deviation the change knowingly leaves behind lands with an entry in the ledger that document names. What a branch leaves behind is therefore the current contract, the recorded distance from it, or both. -1. Changelog is reserved only for changes that are essential meaningful to users. In particular, minor bugfixes should not end up there, let alone refactors that introduce no new features must not be present in the changelog. The entries must stay brief and concise. -1. Bugs and todos should be brief and concise, preferably one sentence per entry. -1. Use American English, in prose and identifiers alike. A name someone else owns keeps the spelling they gave it: `MatchRule.serialise()` is jeepney's, `CancelledError` is the standard library's. - -## Guide - -1. `docs/guide/` is written for someone using the application, not changing it. A page says what a reader can do and how, in the order they would do it; a page organized by control catalogs the application instead of explaining it. -1. A few sentences per feature. Mechanism, file formats and per-widget behavior belong to `docs/development/`, and a `###` inside a guide section is the sign a passage grew into a reference. -1. Write for a reader with no picture of the screen. Name a control by the label the application ships, read from the language file, rather than by where it sits. -1. Write in plain, direct English. Short sentences carrying one fact each, the noun repeated rather than replaced by a pronoun, and a bulleted list wherever the page states several things of one kind. Use everyday verbs — *shows*, *changes*, *opens*, *removes*, *click* — in place of this repository's own vocabulary (*settles*, *holds*, *answers*, *stands for*, *reaches*), which names concepts a reader of the guide has never met. - -## Tools - -1. `docs/tools/` is written for someone running a command that measures _SampleToNES_ or produces examples, in an installed copy or a checkout. A page per tool says what the tool is for, how to run it with no options, what it writes, every custom use, and how it works last. -1. A tool's options are explained on its page. [Tooling](tooling.md) lists the commands in one line each and links to the page. -1. The guide's writing rules hold: plain, direct English, short sentences, and a list wherever the page states several things of one kind. +1. [Writing the documentation](documentation.md) has the rules for every document in the repository: who each one is written for, what belongs in it, and how it reads. Consult it before adding or editing a page. +1. Bugs and to-dos are brief, preferably one sentence per entry. ## Tests 1. A test file mirrors the ownership of the code it exercises. 1. When functionality moves between packages, move its direct unit tests in the same change. -1. **A test whose assertion is a measured duration lives in `tests/benchmarks/`.** The gated suite runs across six workers and under coverage, which multiplies the cost of the code being measured, so those tests run in a pass of their own — serial and uncovered — where the reading is the code's own cost. `make test` runs the covered suite, and `make benchmarks` runs the measured pass. +1. **A test whose assertion is a measured duration lives in `tests/benchmarks/`.** The gated suite runs across several workers and under coverage, which multiplies the cost of the code being measured. Benchmarks run in a pass of their own, serial and uncovered, where the reading is the code's own cost. `make test` runs the covered suite, and `make benchmarks` runs the measured pass. 1. Parametrize tests that share a body, using a test-case dataclass. 1. Test case classes and cases themselves should be defined inside the testing class, unless these objects are shared between test classes. A suite inherits from `BaseTestSuite` and names its case class `TestCase`, which inherits from `BaseRegularTestCase`, or from `BaseAutolabelTestCase` where the case derives its own label. The parametrized argument carries the case as `test_case`. 1. For a multi-step scenario, use a test-scenario suite class — a series of functions with assertions. 1. Prefer fixtures over factories, and define shared fixtures in an appropriate place. -1. **A shipped value is a choice, not a contract.** Defaults, keybinding schemes, palettes and layouts are tuned freely, so a case that restates one turns every adjustment into a test edit. Read the value where it is configured — or from the constant that defines it — and assert the behavior around it: the bound it lies within, the round-trip it survives, the action it answers. Spell a value out only where the value itself is the contract, a file format's constant say, and name that reason in the case. -1. **A case assumes no one platform.** The separators in a path, the ending of a line, the formatting of a number, the order a directory arrives in — these belong to where the suite runs, not to the case. Compare a path with a `Path` rather than with the string POSIX renders it as; the suite runs on Windows too. -1. Values that must match by contract are asserted to match, never hardcoded — e.g. project metadata at creation or after a save/load round-trip is held against its source, never against a version string. -1. Unit tests may mock system boundaries (file I/O, external services, IPC channels), but must not mock the domain logic that is the subject of the test. Integration tests must exercise real computation pipelines against real (synthetically built) data. -1. When a test expectation diverges from the production code's actual behavior, determine which is wrong before acting. A failing test is evidence of a potential bug in the production code unless the test itself is demonstrably incorrect (wrong imports, misread API contract, incorrect fixture). Never silently delete or weaken a test to make it pass. If uncertain, flag the divergence explicitly and ask before changing either side. +1. **A contract stated in prose is pinned by a case.** A behavior a document or a docstring asserts is a promise to whoever reads it next. Write the case that fails once the promise stops holding, and write it where the contract is stated, so the sentence and the assertion move together. +1. **A shipped value is a choice, not a contract.** Defaults, keybinding schemes, palettes and layouts are tuned freely, so a case that restates one turns every adjustment into a test edit. Read the value where it is configured, or from the constant that defines it, and assert the behavior around it: the bound it lies within, the round trip it survives, the action it answers. Spell a value out only where the value itself is the contract, such as a file format's constant, and name that reason in the case. +1. **A case runs on every platform.** The separators in a path, the ending of a line, the formatting of a number and the order a directory arrives in belong to where the suite runs, not to the case. Compare a path with a `Path` and not with the string POSIX renders it as, because the suite runs on Windows too. +1. Values that must match by contract are asserted to match and never hardcoded. For example, project metadata at creation or after a save/load round trip is held against its source and never against a version string. +1. Unit tests may mock system boundaries (file I/O, external services, IPC channels) but not the domain logic that is the subject of the test. Integration tests exercise real computation pipelines against real, synthetically built data. +1. When a test expectation diverges from the production code's actual behavior, determine which is wrong before acting. A failing test is evidence of a potential bug in the production code unless the test itself is demonstrably incorrect (wrong imports, misread API contract, incorrect fixture). Never silently delete or weaken a test to make it pass. If uncertain, flag the divergence and ask before changing either side. diff --git a/docs/development/packages.md b/docs/development/packages.md index 3912a932e..0abdebc3d 100644 --- a/docs/development/packages.md +++ b/docs/development/packages.md @@ -1,14 +1,13 @@ # Package Layers -_SampleToNES_ is one repository holding several packages under `src/`, ordered so that dependencies -run one way. This document states that order, what each package is for, and how the console player -is layered inside it. It is prescriptive: `sampletones_config/boundaries/graphs.yaml` restates these -tables in the form the import-boundary check runs on every commit, and a divergence between this -document and that configuration is itself a defect. +_SampleToNES_ is one repository holding several packages under `src/`, ordered so that dependencies run +one way. This document says what each package is for and why the order runs as it does. Read it when +deciding where a module belongs. `sampletones_config/boundaries/graphs.yaml` declares the order itself, in +the form the import-boundary check runs on every commit. -The layering of `sampletones_application` has its own document, -[`architecture.md`](architecture.md), which the same check enforces. How a long operation reports how far it -has come — inside one process and across the pool's workers — is [`progress.md`](progress.md). +The layering of `sampletones_application` has its own document, [`architecture.md`](architecture.md), which +the same check enforces. [`progress.md`](progress.md) describes how a long operation reports how far it has +come, inside one process and across worker processes. --- @@ -41,106 +40,95 @@ graph TD ENTRY --> SHARED ``` -| Package | Purpose | May import | -|---------|---------|------------| -| `sampletones_shared` | Facts and helpers any package holds: constants, exception families, paths, the logger, the array backend, and the command type the entry and the tools share | — | -| `sampletones_config` | The shipped YAML — layout, palettes, themes, keybindings, language, behavior, deployment, and these boundaries themselves — reached as package data rather than by import | — | -| `sampletones_assets` | The application icons and the bundled fonts, reached as package data | — | -| `sampletones_core` | The reconstruction engine, the project model, playing a song out into instructions, and the tracker export formats | `sampletones_shared` | -| `sampletones_player` | The NES player: the register model, the re-clocking schedule, the 6502 driver and the NSF file | `sampletones_shared`, `sampletones_core` | -| `sampletones_application` | The DearPyGui front end | `sampletones_shared`, `sampletones_core`, `sampletones_player` | -| `sampletones_tools` | Everything a developer runs and the application does not: analytic waveform synthesis, the calibration harness, the driver toolchain and the register trace, the mark the icons are drawn from, the source checks, the synthetic corpus with its sample emitters, and the developer commands that run them | `sampletones_shared`, `sampletones_core`, `sampletones_player`, `sampletones_application` | -| `sampletones` | The command-line entry: the dispatcher, the commands and the startup self-check | `sampletones_shared`, `sampletones_core`, `sampletones_application`, `sampletones_tools` | - -Third-party imports are the package author's own choice and stand outside this table. - -**The tools package is reached from the command line alone.** `sampletones_tools` holds what a -developer runs and the application never imports: the developer commands and the libraries behind -them. `sampletones` is its one importer, appending the developer commands to the user commands, so -the wheel and the bundle carry the tools and no shipped package depends on them. -[Tooling](tooling.md) states what a tool is and what a developer command does in an installed copy. - -**The reconstruction engine stands below the console player.** A reconstruction is produced, saved -and exported to a tracker with `sampletones_player` absent from the process, which is what lets the -player's format move while the engine holds still. The consequence is that an export backend -reaching the console — the seam `sampletones_core/exports/backend.py` describes — is registered -from above rather than from the engine's own registry. - -**A song is played out once, for every reader of it.** Turning an arrangement into the -instruction each channel sounds on each engine tick — the order walked frame by frame, a row's note -column starting a voice, its transpose and volume bending what that voice carries, a voice with a -loop point circling where one without falls silent — is `sampletones_core/performance/`. One -reading answers for both kinds of voice: a sample plays the frames its conversion found for the -channel, a hand-written instrument the frames its envelopes make of it. The sequencer renders -those instructions to audio and the player encodes them into register values, so what a listener -hears and what the console plays are the same walk read two ways rather than two implementations of -one rule. A voice sounded on its own — a preview, an audition at a note a key names — takes the -same two steps a row takes, so it lives there too rather than beside whichever surface asked. - -**Equal temperament sits at the bottom.** The MIDI pitch limits and the A4 reference are -`sampletones_shared/constants/music.py`, and the pitch-to-frequency conversion they govern is -`sampletones_shared/utils/frequencies.py` — so the synthesis package reads them without reaching up -into the engine, and `sampletones_core/utils/frequencies.py` keeps what is the engine's own: the -project's usable pitch range, the noise periods, and the note and period names. +| Package | What it holds | +|---------|---------------| +| `sampletones_shared` | Facts and helpers any package holds: constants, exception families, paths, the logger, the array backend, and the command type the entry and the tools share | +| `sampletones_config` | The shipped YAML — layout, palettes, themes, keybindings, language, behavior, deployment, and these boundaries themselves — reached as package data and not by import | +| `sampletones_assets` | The application icons and the bundled fonts, reached as package data | +| `sampletones_core` | The reconstruction engine, the project model, playing a song out into instructions, and the tracker export formats | +| `sampletones_player` | The NES player: the register model, the re-clocking schedule, the 6502 driver and the NSF file | +| `sampletones_application` | The DearPyGui front end | +| `sampletones_tools` | Everything a developer runs and the application does not: the calibration harness, the driver toolchain and the register trace, the source checks, the synthetic corpus, and the developer commands that run them | +| `sampletones` | The command-line entry: the dispatcher, the commands and the startup self-check | + +**Only the command line reaches the tools package.** `sampletones_tools` has what a developer runs and the +application never imports: the developer commands and the libraries behind them. `sampletones` is its one +importer. It appends the developer commands to the user commands, so the wheel and the bundle carry the +tools and no shipped package depends on them. [Tooling](tooling.md) says what a tool is and what a +developer command does in an installed copy. + +**The reconstruction engine sits below the console player.** A reconstruction is produced, saved and +exported to a tracker with `sampletones_player` absent from the process. That lets the player's format move +while the engine holds still. An export backend that reaches the console (the interface described in +`sampletones_core/exports/backend.py`) is therefore registered from above and not from the engine's own +registry. + +**A song is played out once, for every reader of it.** `sampletones_core/performance/` turns an +arrangement into the instruction each channel sounds on each engine tick. It walks the order frame by +frame. A row's note column starts a voice, and the row's transpose and volume bend what that voice carries. +A voice with a loop point circles, and one without falls silent. One reading answers for both kinds of +voice: a sample plays the frames its conversion found for the channel, and a hand-written instrument plays +the frames its envelopes make of it. + +The sequencer renders those instructions to audio, and the player encodes them into register values. What +a listener hears and what the console plays are therefore the same walk read two ways, and not two +implementations of one rule. A voice sounded on its own, such as a preview or an audition at a note a key +names, takes the same two steps a row takes. It lives in `performance/` too, and not beside whichever +surface asked. + +**Equal temperament sits at the bottom.** The MIDI pitch limits and the A4 reference are in +`sampletones_shared/constants/music.py`, and the pitch-to-frequency conversion they govern is in +`sampletones_shared/utils/frequencies.py`. The synthesis package can therefore read them without reaching +up into the engine. `sampletones_core/utils/frequencies.py` keeps what is the engine's own: the project's +usable pitch range, the noise periods, and the note and period names. --- ## Inside `sampletones_player` -The player divides into units layered the same way, and for the same reason: a register value, a -clock and a song exist independently of the file they are written into or the driver that reads -them. - -| Unit | Purpose | May import | -|------|---------|------------| -| `specification/` | The register addresses, control bits, offsets and address constants the format is written by, one module per subject | — | -| `clock/` | `PlaySchedule` and `FixedPointStep` — the engine ticks one play call advances a stream by | `specification/` | -| `registers/` | The per-tick register values each channel plays, and the four streams together | `specification/` | -| `compression/` | The planes a song separates into, the dictionary its tokens name, and the codec that reads them both ways | `specification/`, `registers/` | -| `song.py` | `Song` — the compressed planes, the timer table, the schedule and the loop point as one value | `clock/`, `registers/`, `compression/` | -| `builder.py` | The song a reconstruction or an export request plays as, its instructions encoded, its planes compressed and its rate scheduled | `song.py`, `registers/`, `clock/`, `compression/` | -| `nsf/` | The song block, the header and the `.nsf` file the console loads | `song.py`, `specification/`, `compression/`, `driver/` | -| `driver/` | The assembled 6502 driver and the addresses its build reports | `specification/` | -| `export/` | `NSFBackend` — the export seam answered in `.nsf` files, writing each request as the program its source states or the one a user chose (`NSFProgram`: channels, repeat, compression scheme and header text), holding the driver every file carries and saying which stage a run is in | `builder.py`, `song.py`, `nsf/`, `driver/`, `compression/` | +The player divides into units layered the same way, and for the same reason: a register value, a clock and +a song exist independently of the file they are written into or the driver that reads them. + +- The specification sits at the bottom. It holds the addresses, control bits and offsets the format is + written by. +- The clock, the per-tick register values and the codec stand on it. +- `Song` gathers what a file carries into one value. +- `builder.py` is the one place a song is made, whatever asked for it. +- `nsf/`, `driver/` and `export/` sit at the top, where a song becomes the file the console loads. + +[The console player](player.md) says what the player is for and what holds it correct. ### The toolchain and the oracle live with the tools -`sampletones_tools/player/assembler/` runs `ca65` and `ld65` over the assembly sources, their -includes and the linker configuration in `sampletones_tools/player/assembly/`, read as package -data, to produce the committed `driver/binary/driver.bin`; `uv run sampletones driver` runs it, -and the tests rebuild the sources and hold the committed image to them wherever cc65 is -installed. `sampletones_tools/player/trace/` holds `RegisterTrace`, what the driver is expected -to write call by call, which the emulator tests hold the assembled driver to. Exporting reads the -assembled binary; the wheel carries the assembly sources beside it, inside the tools package. The -toolchain the build needs is described in [`dependencies.md`](release/dependencies.md). +The driver's assembler and the register trace live in `sampletones_tools/player/`, because exporting needs +neither. The assembler builds the committed `driver/binary/driver.bin` from the assembly sources beside it. +The tests rebuild the sources wherever cc65 is installed and hold the committed image to them. +`RegisterTrace` says what the driver is expected to write, call by call, and the emulator tests hold the +assembled driver to it. Exporting reads the assembled binary. The wheel carries the assembly sources +beside it, inside the tools package. [`dependencies.md`](release/dependencies.md) describes the toolchain +the build needs. --- ## Enforcement -`sampletones_config/boundaries/graphs.yaml` declares both graphs as layer tables — each unit and the -units it may import — and the rule the check runs derives from them: every unit a table leaves out is -out of reach, so an edge is declared before it is taken. The hook audits the whole source tree on -every commit (`uv run sampletones check import-boundary --all`), which means adding an edge to a table is how a new -dependency is opened, and removing one enumerates the work of closing it. +`sampletones_config/boundaries/graphs.yaml` declares both graphs as layer tables: each unit and the units it +may import. The rule the check runs derives from them. Every unit a table leaves out is out of reach, so an +edge is declared before it is taken. Adding an edge to a table opens a new dependency, and removing one +lists the work of closing it. A graph names the repository's own packages, and a third-party import is the +package author's own choice. [Architecture](architecture.md#enforcement) describes the mechanism and the +working idiom that follows from a whole-tree check. -Five token rules hold the shipped packages to the tools edge a second way: a module of +A graph is checked for well-formedness as it is read. A unit reaching a unit the graph leaves undeclared is +refused. So is a graph whose units reach themselves, since a unit's layers state a level only where the +units stand in an order. + +Token rules hold the shipped packages to the tools edge a second way. A module of `sampletones_application`, `sampletones_core`, `sampletones_player`, `sampletones_shared` or -`sampletones_assets` that spells `sampletones_tools` at all is reported, so the edge is closed in -words as well as in imports. - -A graph answers for its own well-formedness as it is read: a unit reaching a unit the graph leaves -undeclared is refused, and so is a graph whose units reach themselves, since a unit's layers state a -level only where the units stand in an order. - -Three parts share the work. `sampletones_config/boundaries/` states what the boundaries are. -`sampletones_tools/checks/boundary/` validates that statement and holds the mechanism — -reading a module line by line, resolving a unit to the modules it owns, deriving a rule from a graph -and reporting what crosses it — beside the source layer the other checks read the tree through, -`sampletones_tools/checks/source/`. `sampletones check import-boundary` runs them over the source -and scripts trees and prints what they find. - -The scripts tree is held to a rule of its own, `boundaries/standalone.yaml`. A bootstrap script -runs on the system interpreter, so it imports the standard library and the scripts tree itself, -and a name in that tree that stands in for a standard-library module is reported too, since the -tree sits on the import path. [Tooling](tooling.md) states the principle. +`sampletones_assets` that spells `sampletones_tools` at all is reported, so the edge is closed in words as +well as in imports. + +A bootstrap script runs on the system interpreter, so the scripts tree has a rule of its own, +`boundaries/standalone.yaml`. A script imports the standard library and the scripts tree itself. A name in +that tree that stands in for a standard-library module is reported too, since the tree sits on the import +path. [Tooling](tooling.md) states the principle. diff --git a/docs/development/player.md b/docs/development/player.md index 06987d89a..08618d413 100644 --- a/docs/development/player.md +++ b/docs/development/player.md @@ -1,81 +1,73 @@ # The console player -This document governs `sampletones_player`: the 6502 driver an exported `.nsf` carries, the -codec that fits a song into the console's program area, and the chain that holds both to -what the application plays. Read it before changing the assembly under -`sampletones_tools/player/assembly/`, anything under `compression/`, or the way a song is built in -`builder.py`. The byte layout the two sides meet on is [the NSF format](../formats/nsf.md); -where the package sits among the others is [package layers](packages.md). - -Everything else _SampleToNES_ exports describes a song to a program that plays it. This one -**is** the program. That single difference sets the whole design: the file has to carry a -player, the player has to fit beside the song in 32 KB, and the song has to be decodable by -a processor that has no multiply. +This document governs `sampletones_player`: the 6502 driver an exported `.nsf` carries, the codec that fits a +song into the console's program area, and the chain that holds both to what the application plays. Read it +before changing the assembly under `sampletones_tools/player/assembly/`, anything under `compression/`, or +the way a song is built in `builder.py`. The byte layout the two sides meet on is +[the NSF format](../formats/nsf.md). [Package layers](packages.md) says where the package sits among the +others. + +Everything else _SampleToNES_ exports describes a song to a program that plays it. This one **is** the +program. That single difference sets the whole design. The file has to carry a player, the player has to +fit beside the song in 32 KB, and the song has to be decodable by a processor with no multiply. ## Principles -**The driver interprets nothing.** Which value silences a channel, how a duty cycle reaches -its bits, how a pitch becomes a period, how the linear counter is held — every one of those -is settled in Python, under `registers/`, where it is testable at a keystroke. What crosses -into assembly is moving bytes to addresses and counting ticks. A rule that would have to be -debugged on a 6502 is a rule in the wrong place. +**Python settles every rule, and the driver moves bytes.** Which value silences a channel, how a duty cycle +reaches its bits, how a pitch becomes a period, how the linear counter is held: each of these is settled +in Python, under `registers/`, where it is testable at a keystroke. What crosses into assembly is moving +bytes to addresses and counting ticks. A rule that would have to be debugged on a 6502 is a rule in the +wrong place. **What a correct driver writes is stated in Python, and the assembly is held to it.** -`RegisterTrace.from_song` says which APU registers a run touches, in what order, call by -call. The assembled driver is run on a 6502 emulator and its writes are compared against -that statement. The oracle is the contract; the assembly is an implementation of it, and -either one being wrong shows up as a difference rather than as a wrong sound. - -**A song is decoded forward, never indexed.** Reading a tick by multiplying its number -bought exactly two things: skipping several ticks in one play call, and jumping to the loop -point. The first is a matter of decoding several ticks in a row; the second the header can -state outright. Giving both up in exchange for compression is what turns a program area -that holds seconds into one that holds minutes. - -**The dictionary is the instrument table.** A song is built by playing samples at rows, so -the shapes its planes repeat are knowable rather than discoverable: each sample offers the -planes it writes, and every row playing it becomes a token naming that entry. Search fills -what the samples leave uncovered. - -**Every layer earns its place on measured ground.** Each stage of the codec can be switched -off on its own, and `uv run sampletones codec report` writes what each one saves across a corpus of -songs. The format's constants are settled from that report rather than from argument. +`RegisterTrace.from_song` says which APU registers a run touches, in what order, call by call. The +assembled driver runs on a 6502 emulator, and its writes are compared against that statement. The oracle +is the contract, and the assembly is an implementation of it. A fault in either one shows up as a +difference and not as a wrong sound. + +**A song is decoded forward.** Reading a tick by multiplying its number would allow two things: skipping +several ticks in one play call, and jumping to the loop point. Decoding several ticks in a row covers the +first, and the header can state the second outright. Giving up indexing is what turns a program area that +holds seconds into one that holds minutes. + +**The dictionary is the instrument table.** A song is built by playing samples at rows, so the shapes its +planes repeat are knowable in advance. Each sample offers the planes it writes, and every row playing it +becomes a token naming that entry. Search fills what the samples leave uncovered. + +**Every layer earns its place on measured ground.** Each stage of the codec can be switched off on its +own, and `uv run sampletones codec report` writes what each one saves across a corpus of songs. The +format's constants are settled from that report and not from argument. **A change to the codec is measured before it is built.** `uv run sampletones codec study` reads the -projects and stems named on its command line, encodes every song under every candidate change, -and writes the sizes, the times, a verdict per candidate and the manifest that repeats the run -under `Documents/SampleToNES/compression`. A candidate is one of two things. A new way of choosing -tokens is encoded and played back by the production codec itself. A new token grammar is -priced in bytes by a study parser, which first has to reproduce the production parser's -bytes on today's grammar. The rule is printed in the report: a candidate earns a production -layer when it saves 3% over the projects or 5% over the reconstructions and grows no song by -more than 1%. The study lives under `sampletones_tools/codec/study`, outside the shipped -packages. +projects and stems named on its command line, encodes every song under every candidate change, and writes +the sizes, the times, a verdict per candidate and the manifest that repeats the run under +`Documents/SampleToNES/compression`. A candidate is one of two things. A new way of choosing tokens is +encoded and played back by the production codec itself. A new token grammar is priced in bytes by a study +parser, which first has to reproduce the production parser's bytes on today's grammar. The report prints +the rule that gives a candidate a production layer: it must save a set share of the projects' or of the +reconstructions' bytes, and it must not grow any song beyond a set margin. The study lives under +`sampletones_tools/codec/study`, outside the shipped packages. ## The song a file carries -`Song` is the compressed song: the dictionary, one token stream per plane, the timer table, -the clock and the loop point. The register values every channel writes are read back out of -the streams on demand, so a trace, a writer and a test all speak to the compressed song -without knowing it is one. - -A song is built three ways, and the difference between them is only where the ticks come -from: `song_from_reconstruction` sounds a reconstruction's own instructions, -`song_from_sample` sounds the slices of an export request, and `song_from_project` plays a -whole arrangement out row by row through the same walk the sequencer sounds a song with. -The last of those is the one that seeds the dictionary from the project's samples. - -**A program states what an export chooses.** `NSFProgram` holds the channels a song sounds, -the tick it returns to, the `CompressionScheme` it is written with and the header text, and -the builders take the first three explicitly. `NSFProgram.for_project` and -`NSFProgram.for_sample` state what an export writes when nobody chose otherwise, and the -export dialog opens on the same values. `NSFBackend` answers the export seam either with those -stated programs or, through `choosing`, with one a user settled, so the service that runs an +`Song` is the compressed song: the dictionary, one token stream per plane, the timer table, the clock and +the loop point. The register values every channel writes are read back out of the streams on demand, so a +trace, a writer and a test all speak to the compressed song without knowing it is one. + +A song is built from a reconstruction's own instructions, from the slices of an export request, or from a +whole project. The three differ only in where the ticks come from. A project is played out row by row +through the same walk the sequencer sounds a song with, and it is the case that seeds the dictionary from +the project's samples. + +**A program states what an export chooses.** `NSFProgram` holds the channels a song sounds, the tick it +returns to, the `CompressionScheme` it is written with and the header text. The builders take the first +three explicitly. The defaults an export writes when nobody chose otherwise are stated once, and the export +dialog opens on the same values. A user's choice replaces those defaults, so the service that runs an export stays free of anything the format decides. -A project carries no tuning of its own — each sample was reconstructed against one — so the -samples state it by agreeing on it, and a project whose samples disagree is refused rather -than sounded half in tune. +A project has no tuning of its own, because each sample was reconstructed against one. The samples +therefore state the tuning by agreeing on it. A project whose samples disagree is refused, so nothing +sounds half in tune. ## The codec @@ -97,10 +89,23 @@ from that pitch. It saves a byte a tick directly, but the reason it matters is t cannot be transposed and an index can: the same figure played at several pitches is several copies in timer space and one entry plus a shift in index space. +**A plane repeats from its own byte.** Four of the planes write a byte the APU only +partly reads — two bits a pulse control byte wants set, the top nibble of a noise control +byte, three bits above a noise period — and those spare bits carry the ticks the value +repeats for. A rest costs one byte however long it lasts, and the planes that rest longest +are exactly the ones with room to say so. The division follows from the register, so the +block states none of it and the driver knows it by plane. + +**The triangle names its silence in the pitch it plays.** The channel sounds at one level, +so the index its value plane carries is enough to say whether it sounds at all: an index +above every pitch the table holds silences the linear counter. That spares a whole plane, +in the block and on the tick path both. + **Tokens.** A plane is written as holds, literals and phrase plays — the encoding is in [the format document](../formats/nsf.md#b4-the-token-streams). What matters here is that a token's count is a duration rather than a length, so one dictionary entry serves a figure -however long it is held and whatever pitch it is played at. +however long it is held and whatever pitch it is played at, and that a phrase may state +the count its tokens play it at most often so those tokens carry none. **The cheapest reading, not a greedy one.** A plane is parsed as a shortest path: every way of covering a tick is an edge priced in the bytes its token takes, and the cheapest path @@ -124,9 +129,6 @@ true fraction of the song. ## The driver -The driver is three sources: the entry points and the play call in `driver.s`, the clock in -`clock.s`, and the plane decoders in `channels.s`. - **The clock steps a tick at a time.** A play call adds the header's step to an accumulator and reads the whole ticks off the top; the driver then moves the clock on by one tick at a time, and each step answers what the channels are to do with it — play it, play it from the @@ -139,7 +141,8 @@ seeded into page zero, which no song occupies, and the advance passes it by, so the zero every plane starts from. A bend plane is stepped only on a tick its channel's new value flags, since it holds a value for those ticks alone. A plane's state carries where its next token lies, where in a phrase body it stands, how much of that body is left, how much of the -current token is left, the value it last played, and the shift it is playing at. The three +current token is left, the symbol it last played, the ticks that symbol still covers, the +bits its own byte counts in, and the shift it is playing at. The three kinds of token fold into that one shape — a hold is a phrase of no bytes, a literal is a phrase whose bytes lie inline behind its opcode — so playing a tick is the same handful of instructions whichever token is standing. @@ -149,46 +152,38 @@ is a base offset held in `X`, the way a channel's register base is, so every pla is one routine called again. The state lives in zero page, well inside what the driver leaves free, and the linker configuration keeps the two-segment memory model an NSF loads. -**The one sum the driver performs is the bend.** A tone channel's value plane resolves to a -divider through the timer table, and on a flagged tick its bend plane states the steps the -tick stands away from it — sign-extended and added across both halves of the timer, with the -high half reaching the register only where it changed. An unflagged tick adds nothing. Everything that keeps the sum in range is -settled in Python, so what crosses into assembly stays a byte moved and a carry followed. +**The driver's arithmetic is the bend and the repeat.** A tone channel's value plane +resolves to a divider through the timer table, and on a flagged tick its bend plane states +the steps the tick stands away from it — sign-extended and added across both halves of the +timer, with the high half reaching the register only where it changed. An unflagged tick +adds nothing. The other is a subtraction: a tick a symbol still covers steps the plane's +count down and returns, which makes the common tick cheaper than reaching a new symbol. +Everything that keeps either in range is settled in Python, so what crosses into assembly +stays a byte moved and a carry followed. ## How it is verified -The chain runs from the register values upward, and each link is held on its own: - -| Level | How | -|---|---| -| The codec is lossless | every encoding decodes to the planes it was written from, over a corpus | -| The codec is safe | a plane the codec finds nothing in stays within its literal bound | -| The ratio | `uv run sampletones codec report` — bytes per tick and ticks that fit, per layer | -| What a change would save | `uv run sampletones codec study` — the projects and stems it is given, under every candidate change, with a verdict each | -| The byte layout | a hand-built song serializes to expected bytes | -| The assembly agrees with the specification | the include's equates are read and compared field by field | -| The driver behaves | the assembled image on a 6502 emulator against `RegisterTrace.from_song`, over several rates and over songs that repeat | -| The driver's arithmetic | a song stating a bend plane outright, and bent frames exported end to end, each held to the divider the sequencer sounds every tick at | -| The audio | a captured trace re-rendered against the reconstruction's own approximation | -| The whole export | a project exported, played on the emulator, and read back as the instructions the sequencer sounds | -| Listening | `uv run sampletones nsf samples -o build/nsf` then `uv run sampletones nsf render --directory build/nsf`, or any NSF player | -| Speed | `make benchmarks` — the encoder's own cost on the shapes that scale worst | - -The audio comparison is the one that catches a mistake the trace would let through: the -trace says the right registers were written, and the render says the result is the waveform -the reconstruction was built as. +The chain runs from the register values upward, and each link is held on its own: the codec against its +golden decoder over a corpus, the byte layout against a hand-built song, the assembly's equates against +`specification/`, and the assembled image on a 6502 emulator against `RegisterTrace.from_song`. On top sits +a whole export: a project exported, played on the emulator, and read back as the instructions the sequencer +sounds. + +One more check covers what the others cannot. The trace says the right registers were written. The audio +comparison re-renders a captured trace against the reconstruction's own approximation and says the result +is the waveform the reconstruction was built as. ## Building the driver -`uv run sampletones driver` assembles the sources under `sampletones_tools/player/assembly/` -with cc65 and writes `sampletones_player/driver/binary/driver.bin`, which is committed — -exporting an `.nsf` needs no assembler, and the application ships the binary alone. +The driver binary is committed as `sampletones_player/driver/binary/driver.bin`, so exporting an +`.nsf` needs no assembler and the application ships the binary alone. `uv run sampletones driver` +rebuilds it from the sources under `sampletones_tools/player/assembly/` and prints the layout the +build produced. -The link line names our own configuration and our own object files, with the CPU stated -outright. That is the guardrail that keeps the shipped image entirely ours: reaching for a -cc65 target or library would place that project's start-up code and runtime in the bytes the -package distributes. A build also holds the linker's own labels against the addresses the -exporter states without one, so the committed image and the header describing it cannot -drift apart. +The link line names this project's own configuration and object files and states the CPU outright. That +keeps the shipped image entirely ours: a cc65 target or library would place that project's start-up code +and runtime in the bytes the package distributes. A build also holds the linker's own labels against the +addresses the exporter states without one, so the committed image and the header describing it stay in +step. Installing cc65 is covered in [dependencies](release/dependencies.md). diff --git a/docs/development/progress.md b/docs/development/progress.md index 4f660d08e..2be200d28 100644 --- a/docs/development/progress.md +++ b/docs/development/progress.md @@ -4,12 +4,11 @@ This document governs how a long operation says how far it has come, and how tha the reader watching it. Consult it when adding an operation that takes long enough to be watched, when changing what one reports, or when the report has to cross a process boundary. -The subsystem spans three packages: the operations that report live in `sampletones_core` and -`sampletones_player`, the line a report crosses between processes is -`sampletones_core/parallelization/channel/`, and the layer that draws it is -`sampletones_application/services/` and `logic/`. Layering between those packages is -[`packages.md`](packages.md); the application's own layers are -[`architecture.md`](architecture.md). +The subsystem spans several packages. The operations that report live in `sampletones_core` and +`sampletones_player`. The line a report crosses between processes is +`sampletones_core/parallelization/channel/`. The layer that draws it is `sampletones_application/services/` +and `logic/`. [`packages.md`](packages.md) describes the layering between those packages, and +[`architecture.md`](architecture.md) describes the application's own layers. --- @@ -24,169 +23,108 @@ answering whether the run goes on. ExportReporter = Callable[[ExportProgress], bool] ``` -Each domain names its own progress type — `ExportProgress`, `WalkProgress`, `CodecProgress`, -`ReconstructionProgress` — and its own `announce`, which builds that type, offers it, and raises -`OperationCanceled` where the answer is no. One reporter therefore carries both directions: an -operation is watched and withdrawn over the same line, and a caller that wants neither passes -`silent_reporter` (`sampletones_shared/utils/progress.py`) and hears the run through to its end. +Each domain names its own progress type (`ExportProgress`, `WalkProgress`, `CodecProgress`, +`ReconstructionProgress`) and its own `announce`. `announce` builds that type, offers it, and raises +`OperationCanceled` where the answer is no. One reporter therefore carries both directions: an operation is +watched and withdrawn over the same line. A caller that wants neither passes `silent_reporter` +(`sampletones_shared/utils/progress.py`) and hears the run through to its end. ### 2. A stage names the unit its counts are in -A run passes through stages counting in units of their own — a song's ticks, a dictionary's bytes, -a recording's frames, a batch's files. A report therefore names its stage, and what the counts mean -is read from that name. `ExportStage` and `ReconstructionStage` are the two vocabularies today. +A run passes through stages counting in units of their own: a song's ticks, a dictionary's bytes, a +recording's frames, a batch's files. A report therefore names its stage, and the name says what the counts +mean. `ExportStage` and `ReconstructionStage` are the vocabularies. -Where the stages differ in what they cost, the enum states what each is worth against the others -(`STAGE_WEIGHTS`), so one reading spans a run that changes what it is counting several times over. -The weights are approximations measured over whole runs, justified by what they achieve: a bar that -tracks the time a run actually takes. They are counts rather than fractions, and a reading divides -them once at the point of use, which is what lets a finished run arrive exactly at its end. +Where the stages differ in what they cost, the enum says what each is worth against the others +(`STAGE_WEIGHTS`), so one reading spans a run that changes what it is counting several times over. The +weights are approximations measured over whole runs, and they are justified by what they achieve: a bar +that tracks the time a run actually takes. They are counts and not fractions. A reading divides them once +at the point of use, so a finished run arrives exactly at its end. ### 3. A report is filed as often as the run moves, and carried as often as it is worth reading -An operation announces every step it makes — every frame, every row — because that is what it -knows. Deciding how often that is worth passing on belongs to whoever carries it: `ReportRate` -(`sampletones_shared/utils/progress.py`) spaces reports over `PROGRESS_STEPS` whatever the stage -counts in, always takes a stage's first reading and its last, and treats a count that falls as a -step the same way one that rises. +An operation announces every step it makes, every frame and every row, because that is what it knows. +Whoever carries the report decides how often that is worth passing on. `ReportRate` +(`sampletones_shared/utils/progress.py`) spaces reports over `PROGRESS_STEPS` whatever the stage counts in. +It always takes a stage's first reading and its last, and it treats a count that falls as a step, the same +as one that rises. -Both carriers use it: `StageProgress` in the application services, and `JobReporter` in the -conversion, which throttles on the worker side so the line between processes carries only what a -bar can be redrawn at. +Both carriers use it: `StageProgress` in the application services, and `JobReporter` in the conversion. +`JobReporter` throttles on the worker side, so the line between processes carries only what a bar can be +redrawn at. -### 4. A run reads in one unit, and the run's own size picks which +### 4. A run reports in items when it has many, and in the part done of one item when it has one -An operation counts the items it is measured in — files, samples, jobs. Where it is measured in -many, that count is the reading: the items are what a reader recognizes, and a run holding several -of them under way at once has no one item to follow. Where it is measured in one, the count stands -at nothing out of one for the run's whole length, so the reading is the part of that one item done, -which the item reports itself. +An operation counts the items it is measured in: files, samples, jobs. Where it is measured in many items, +that count is the reading. The items are what a reader recognizes, and a run with several under way at +once has no one item to follow. Where it is measured in one item, the count stands at nothing out of one +for the run's whole length. The reading is then the part of that one item done, which the item reports +itself. -Both layers carry the same pair, and each answers through `is_single` which of the two it is: +Both layers carry the same pair, and each tells through `is_single` which of the two it is: | Layer | Type | The counts | The work under way | |-------|------|-----------|--------------------| | Core | `TaskProgress` | `completed` / `total` | `steps`, one per running task | | Application | `ServiceProgress` | `completed` / `total` | `partial`, in items | -and both derive `fraction` from them. The layer that turns a run's account into a result states the -unit that run reads in — `ConversionService` does it for a conversion — so the count a status line -prints, the stage it names, the bar it draws and the estimate beside it all answer in that one unit. +Both derive `fraction` from them. The layer that turns a run's account into a result states the unit that +run reads in (`ConversionService` does it for a conversion). The count a status line prints, the stage it +names, the bar it draws and the estimate beside it therefore all answer in that one unit. The logic layer +reads them under `logic/main/converter/`, and `ETAEstimator` (`parallelization/progress.py`) makes the +estimate from what a run has covered. ### 5. A task reaching another process reports over a channel Where an operation runs in a worker process, its reports reach the run over a `ProgressChannel` -(`sampletones_core/parallelization/channel/`). The channel carries both directions of principle 1 -across the boundary, and callers depend on the Protocol rather than on any one way of crossing it. +(`sampletones_core/parallelization/channel/`). The channel carries both directions of principle 1 across +the boundary, and callers depend on the Protocol and not on any one way of crossing it. -```python -class ProgressChannel(Protocol): - def reporter(self, index: int) -> StepReporter: ... - def poll(self, timeout: float) -> Optional[TaskReport]: ... - def withdraw(self) -> None: ... - def close(self) -> None: ... -``` - -`ProcessProgressChannel` is the implementation. A manager stands beside the pool and owns both -ends — a queue reports travel up and a flag a withdrawal travels down — under the same spawn -context the pool's workers run in. Each end reaches a task as an ordinary value it is built with, -so a worker started as a fresh interpreter reconnects to them on its own, which is what makes one -channel serve every platform. +`ProcessProgressChannel` is the implementation. A manager stands beside the pool and owns both ends: a +queue that reports travel up, and a flag that a withdrawal travels down. Both live under the same spawn +context the pool's workers run in. Each end reaches a task as an ordinary value it is built with, so a +worker started as a fresh interpreter reconnects on its own. That makes one channel serve every platform. --- ## Mechanics -### The conversation a run has with its tasks - -```mermaid -sequenceDiagram - participant REC as Reconstructor - participant JOB as JobReporter - participant CH as ProcessProgressChannel - participant PUMP as ProgressPump - participant RUN as TaskProcessor - participant SVC as ConversionService - - Note over REC,JOB: worker process - REC->>JOB: announce(MATCHING, frame, frames) - JOB->>CH: TaskStep, where a step is due - CH-->>JOB: whether the run goes on - Note over CH,PUMP: process boundary - PUMP->>CH: poll, then drain what waits - PUMP->>RUN: TaskSteps.record + notify once - RUN->>SVC: TaskProgress(completed, total, steps) - SVC->>SVC: ServiceProgress(partial=…, current_item=ConversionItem) -``` - -A run's monitor thread waits on the results the pool hands back, so reading the channel belongs -to a thread of its own: `ProgressPump`. It takes everything already waiting in one turn and -announces once, which holds the announcements to the rate it reads at however many steps the tasks -file in between. - -### Who owns what - -| Concern | Owner | -|---------|-------| -| The reporter shape, the silent one, and the spacing between reports | `sampletones_shared/utils/progress.py` | -| A reconstruction's stages, their shares, and its own `announce` | `sampletones_core/reconstructions/{stage,progress}.py` | -| Weighing a job's stage and carrying it to the run | `JobReporter` (`reconstructions/converter/progress.py`) | -| The line between a run and its tasks | `sampletones_core/parallelization/channel/` | -| Where the running tasks stand, and dropping a finished one | `TaskSteps` (`parallelization/steps.py`) | -| Ending the pool, opening the channel, reading it, and reaping it | `TaskProcessor` (`parallelization/processor.py`) | -| How long a run has left, from what it has covered | `ETAEstimator` (`parallelization/progress.py`) | -| Turning a run's account into a result the application reads, in the unit its size picks | `ConversionService` (`services/conversion/`) | -| The bar, and the taskbar the run also reports to | `ConversionRun` (`logic/main/converter/run.py`) | -| The status line and the stage's name | `ConverterMessages` (`logic/main/converter/messages.py`) | +**Reading the channel belongs to a thread of its own.** A run's monitor thread waits on the results the +pool hands back, so a separate thread (`ProgressPump`) reads the channel. It takes everything already +waiting in one turn and announces once. That holds the announcements to the rate it reads at, however many +steps the tasks file in between. ### A finished task stays finished -A task's reports travel a line the run reads at its own pace, so one filed before its result -arrived may be read after it. `TaskSteps` records the tasks the run has counted and lets their -later reports go, which is what keeps a task's own progress and the run's completed count from -describing the same work twice — and keeps the reading inside the run it describes. +A task's reports travel a line the run reads at its own pace, so a report filed before a task's result +arrived may be read after it. The run records the tasks it has counted (`TaskSteps`) and lets their later +reports go. That keeps a task's own progress and the run's completed count from describing the same work +twice, and it keeps the reading inside the run it describes. ### The pool's and the channel's lifetime -The run's monitor thread is the one owner of its pool. However the run ends, the monitor ends the -pool — a finished pool winds down, a canceled or failed one is stopped — reaps its workers, closes -the channel, and only then announces the outcome. `cancel()` withdraws the run and leaves the rest to -the monitor; `shutdown()` cancels a run still going and returns once the monitor has finished. The -owner of a run (`InstructionsLibraryManager`, `ConversionService`) lets it go only after -`shutdown()` returns, so an operation reads as active until no worker of it is left. - -The channel is a process of its own, opened on the first task that asks for a line and closed after -the pool, since the workers hold proxies to it. A run whose tasks report nothing never opens one. +The run's monitor thread (`TaskProcessor`) is the one owner of its pool. However the run ends, the monitor +ends the pool: a finished pool winds down, and a canceled or failed one is stopped. It then reaps the +workers, closes the channel, and only then announces the outcome. -### Adding an operation that reports +`cancel()` withdraws the run and leaves the rest to the monitor. `shutdown()` cancels a run still going and +returns once the monitor has finished. The owner of a run (`InstructionsLibraryManager`, +`ConversionService`) lets it go only after `shutdown()` returns, so an operation reads as active until no +worker of it is left. -1. Give the domain a stage enum and a progress type, with an `announce` beside them, following - `sampletones_core/exports/progress.py`. -2. Thread the reporter through the calls that know how far the work has come, and announce every - step they make. -3. Where the work runs in this process, hand it a carrier that throttles and emits — `StageProgress` - for a service. Where it runs in a worker, ask `TaskProcessor._task_reporter` for a line and hand - the task a reporter built on it. -4. Read the run through `fraction`, and name the stage from `LanguageManager` in the logic layer. +The channel is a process of its own. It opens on the first task that asks for a line and closes after the +pool, since the workers hold proxies to it. A run whose tasks report nothing never opens one. --- ## Testing -Progress is one path with two halves, and each is tested where it is cheap to test: - -| What | Where | -|------|-------| -| A stage's share, and a run read as a fraction | `tests/unit/sampletones_core/reconstructions/test_progress.py` | -| The spacing between reports | `tests/unit/sampletones_shared/utils/test_progress.py` | -| Where the running tasks stand, and a late report | `tests/unit/sampletones_core/parallelization/test_steps.py` | -| A job's stage weighed and throttled | `tests/unit/sampletones_core/reconstructions/converter/test_progress.py` | -| The real pipeline reporting its stages, in this process | `tests/integration/reconstruction/test_conversion_jobs.py` | -| The line carrying steps and a withdrawal between real processes | `tests/integration/sampletones_core/parallelization/test_progress_channel.py` | -| A conversion reporting itself end to end | `tests/integration/reconstruction/test_conversion_progress.py` | - -The two cross-process suites run real worker processes and stand something cheap in for the work — -counting in one, a walk through the stages in the other — so what they measure is the wiring rather -than a reconstruction. Both hold their worker at a chosen point until the test has taken the reading -it is asserting on (`tests/suite/release.py`), so an assertion about work under way is made while -that work is provably under way rather than resting on the scheduler. +Progress is one path with two halves, and each is tested where it is cheap to test. A stage's share, the +spacing between reports and a late report are unit-tested beside the code that decides them. The pipeline +reporting its stages and the line carrying a report between processes are held by integration suites. + +The two cross-process suites run real worker processes and put something cheap in place of the work: +counting in one, a walk through the stages in the other. They measure the wiring and not a reconstruction. +Both hold their worker at a chosen point until the test has taken the reading it asserts on +(`tests/suite/release.py`). An assertion about work under way is therefore made while that work is provably +under way, and does not rest on the scheduler. diff --git a/docs/development/release/compatibility.md b/docs/development/release/compatibility.md index d42da007f..42521effa 100644 --- a/docs/development/release/compatibility.md +++ b/docs/development/release/compatibility.md @@ -1,14 +1,10 @@ # Data Compatibility -This document governs the data versions of the stored formats of _SampleToNES_: -reconstruction files (`.stn`), instruction libraries (`.ins`), and project -documents (`project.json`). Reconstructions and projects are upgraded; libraries -are rebuilt. Consult it when changing a serialized shape, adding a format -version, or diagnosing a file that loads as incompatible. +This document governs the data versions of the stored formats of _SampleToNES_: reconstruction files (`.stn`), instruction libraries (`.ins`) and project documents (`project.json`). Reconstructions and projects are upgraded, and libraries are rebuilt. Consult it when changing a serialized shape, adding a format version, or diagnosing a file that loads as incompatible. -The upgrades live in `sampletones_core/compatibility` and run at the load -boundary of each format, before deserialization. The formats' own documents -describe their stored shape and versioning: +Three terms recur. A **payload** is a file's serialized content before any model reads it. A **step** is one upgrade from a version to the next. A **chain** is the steps of one format, in order. + +The upgrades live in `sampletones_core/compatibility` and run at the load boundary of each format, before deserialization. The formats' own documents describe their stored shape and versioning: - [`formats/reconstructions.md`](../../formats/reconstructions.md) - [`formats/instruction-libraries.md`](../../formats/instruction-libraries.md) @@ -18,194 +14,50 @@ describe their stored shape and versioning: ### A format reads and writes one data version -Each format states the single data version this build produces, held in -`SAMPLETONES_LIBRARY_DATA_VERSION`, `SAMPLETONES_RECONSTRUCTION_DATA_VERSION`, -and `SAMPLETONES_PROJECT_DATA_VERSION` (`sampletones_shared/application.py`). -The version travels inside every stored file, and the format's load contract -holds each file to it: `MetadataContract` for the binary formats, the -`format_version` check for projects. +Each format has one data version this build produces, held in `SAMPLETONES_LIBRARY_DATA_VERSION`, `SAMPLETONES_RECONSTRUCTION_DATA_VERSION` and `SAMPLETONES_PROJECT_DATA_VERSION` (`sampletones_shared/application.py`). The version travels inside every stored file, and the format's load contract holds each file to it: `MetadataContract` for the binary formats, and the `format_version` check for projects. ### An upgrade is one version step -A stored shape changes in small, named steps. Each step is a `VersionUpdate`: -the version the payload reads at, the version it writes after the transform, and -the transform itself. The steps of one format form a chain, registered in -`compatibility//__init__.py`, and each step lives in a module named -after the version it writes — `compatibility/reconstruction/v2_2.py` carries -the step that writes reconstruction data version 2.2. - -That step shows the shape a whole step takes, and it is four operations wide. It -lets the stored audio go, since a 2.2 reconstruction renders its channels from -the instructions it keeps; it names each stored stream by the channel that plays -it, where 2.1 named it by its generator; it stamps the configuration the file -carries with the version its shape now matches, which the load contract reads -alongside the outer metadata; and it states the record of the one recording the -file answered to — the channels the run handed out and the level it drove them -at, the file it was read from, and the frame-by-frame account of what it holds, -where a frame that sounds answers to the recording and a silent one answers to -rest. - -**A step owes only what a release wrote.** The shape a payload reaches is what -the model reads, and a model reads the fields it declares: a key the current -shape has no field for is never looked at, so a step that renames or removes one -is doing nothing. The 2.1 step therefore leaves the retired configuration -settings where they stand and spends its effort on the sections the document is -read from. +A stored shape changes in small, named steps. Each step is a `VersionUpdate`: the version the payload reads at, the version it writes after the transform, and the transform itself. The steps of one format form a chain, registered in `compatibility//__init__.py`. Each step lives in a module named after the version it writes: `compatibility/reconstruction/v2_2.py` has the step that writes reconstruction data version 2.2. + +**A step reshapes only what the model reads.** A model reads the fields it declares, so a key the current shape has no field for is never looked at. A step that renames or removes such a key does nothing. Steps leave those keys where they stand and spend their effort on the sections the document is read from. ### A version belongs to a release -The version a format writes moves once per release. Between releases that -version is still being written: every file carrying it was written by a working -tree, so a further change to the stored shape extends the step already pending -rather than adding a second one, and that step widens to carry the whole -distance from the version the last release shipped. What a user's files travel -is therefore one step per release, and `git show :src/sampletones_shared/application.py` -names the version their files stand at. +The version a format writes moves once per release. Between releases that version is still being written: every file carrying it was written by a working tree. A further change to the stored shape therefore extends the step already pending and does not add a second one. That step widens to carry the whole distance from the version the last release shipped. What a user's files travel is one step per release, and `git show :src/sampletones_shared/application.py` names the version their files stand at. ### A chain applies whole or not at all -An upgrade runs only when the registered steps form a complete path from the -file's version to the version this build writes. A file whose version no chain -reaches comes back unchanged, and the format's load contract refuses it, exactly -as it refuses any version this build does not support. A partial path leaves a -file entirely untouched. +An upgrade runs only when the registered steps form a complete path from the file's version to the version this build writes. A file whose version no chain reaches comes back unchanged, and the format's load contract refuses it, as it refuses any version this build does not support. A partial path leaves a file entirely untouched. ### Upgrades run on the raw payload -Upgrades apply to the serialized payload before any model sees it: the msgpack -mapping for `.stn`, the JSON document for `project.json`. The -transform steps reshape that payload — renaming the fields whose names changed -between versions, and adjusting the values they hold where the shape demands it. +Upgrades apply to the serialized payload before any model sees it: the msgpack mapping for `.stn`, the JSON document for `project.json`. A transform reshapes that payload. It renames the fields whose names changed between versions and adjusts the values they hold where the shape demands it. ### A library is rebuilt from its settings -A library is derived data: its `InstructionsLibraryConfig` and the generators -determine it wholly, and generating one costs about what measuring its entries -costs. A library written at another version is therefore rebuilt: -`LibraryState` reads the version from the metadata that leads the file, and a -file the load contract would refuse reads as out of date. A conversion — in the -application, headless, or in calibration — rebuilds the library it needs -unprompted, and opening one from the _Instructions_ tab asks first. -Reconstructions and projects carry their own configuration, so what a user made -stands apart from the libraries it was converted with. - -The library version names what generation produces, so any change to the -generators or to feature extraction bumps it, and the bump alone carries the -change to every stored library. The corpus keeps libraries all the same: a -library a release wrote is held to reading as out of date, and a library -archived at the version this build writes is held to what this build generates -for the same tones. +A library is derived data: its `InstructionsLibraryConfig` and the generators determine it wholly, and generating one costs about what measuring its entries costs. A library written at another version is therefore rebuilt. `LibraryState` reads the version from the metadata that leads the file, and a file the load contract would refuse reads as out of date. A conversion, whether in the application, headless or in calibration, rebuilds the library it needs without asking. Opening one from the _Instructions_ tab asks first. Reconstructions and projects carry their own configuration, so what a user made stands apart from the libraries it was converted with. -### A completed upgrade stamps the version it reached +The library version names what generation produces. Any change to the generators or to feature extraction bumps it, and the bump alone carries the change to every stored library. The corpus keeps libraries all the same. A library a release wrote is held to reading as out of date, and a library archived at the version this build writes is held to what this build generates for the same tones. -A payload whose chain ran carries the new version in the same field it declares -it with, so the file states the version its shape now matches and a later save -writes that version. The load path leaves the bytes of every other payload -untouched. - -## Mechanics - -### Package layout - -- `compatibility/kind.py` — `ObjectKind`, the format an upgrade belongs to - (`LIBRARY`, `RECONSTRUCTION`, `PROJECT`). -- `compatibility/update.py` — `VersionUpdate`, one named version step. -- `compatibility/upgrade.py` — the engine: `upgrade`, `upgrade_binary`, - `upgrade_json`, and the per-format registries `CURRENT_VERSIONS` and `UPDATES`. -- `compatibility//__init__.py` — that format's `UPDATES` tuple. The - reconstruction chain currently holds the 2.1→2.2 step - (`compatibility/reconstruction/v2_2.py`), and the project chain the 1.0→1.1 step - (`compatibility/project/v1_1.py`). Libraries have no chain. - -### Version fields - -- `.stn` — `metadata.reconstruction_data_version` -- `.ins` — `metadata.library_data_version` -- `project.json` — `format_version` at the document root - -### Load boundaries - -`Reconstruction.deserialize_data` passes its payload through `upgrade_binary`; -`ProjectContainer.load` passes the document -through `upgrade_json`. Each wrapper parses the payload, reads the format's -version field, runs the chain, and re-encodes the upgraded payload. A payload -that stays as it is — no chain applies, no version field, or a payload that does -not parse to a mapping — returns as the same bytes, so the load path behaves for -it exactly as it did before the upgrades existed. A file whose version no chain reaches arrives at the format's load contract -unchanged, which refuses it with the format's `Incompatible*VersionError`, as it -always did. - -### Adding an upgrade - -Read the format's version constant against the one the last release shipped, and -take whichever route that comparison names. A library change takes the version -bump alone: bump `SAMPLETONES_LIBRARY_DATA_VERSION` where it still stands at the -shipped version, and leave it where it already stands ahead. - -**The constant stands where the release left it.** The change opens a new step: - -1. Bump the format's version constant in `sampletones_shared/application.py`. -2. Add the step module named after the new version — e.g. - `compatibility/reconstruction/v2_2.py` — with a transform that takes the - payload at the previous version and returns it at the new one. -3. Append the step to the format's `UPDATES` tuple. -4. Cover the step with unit tests under - `tests/unit/sampletones_core/compatibility/`, and hold it to the archived file - its base version names (see [The corpus](#the-corpus)). - -**The constant already stands ahead of the release.** The pending step is the -one to widen: fold the new transform into the module named after that version, -state the whole step from the shipped version in its docstring, and extend its -tests to cover what was added. The version constant stays where it is. - -The engine stamps the new version once the chain runs, so a step module declares -only its own transform. - -## The corpus - -A step tested against a payload the test builds itself is held only to the fields -whoever wrote the test thought to name. `tests/data/compatibility` keeps one -stored document per format per shipped data version, written by the build that -shipped it, and `tests/integration/compatibility` opens each one through that -format's own load entry point and holds the loaded model to the current shape. -The corpus states its own rules in `tests/data/compatibility/README.md`. - -Two of those tests hold the corpus itself together: a file states the version its -name says, and every registered step reads a version the corpus keeps. The second -is what makes opening a step and archiving the file it reads one act rather than -two. +### A completed upgrade stamps the version it reached -### Archiving a release +A payload whose chain ran carries the new version in the same field it declares it with, so the file says the version its shape now matches and a later save writes that version. The load path leaves the bytes of every other payload as they are. -From a checkout of the release, once the version constants have moved: +## Adding an upgrade -``` -uv run sampletones compatibility -``` +Compare the format's version constant with the one the last release shipped, and take the route that comparison names. -A version already archived stands as it was written, since replacing a file would -restate history under a name that already means something. +**The constant stands where the release left it.** The change opens a new step. Bump the constant, add the step module named after the new version with a transform from the previous version to it, and append the step to that format's chain. Cover the step with unit tests and against the archived file its base version names. The archived file is what makes the tests trustworthy: a step tested only on a payload the test builds is held to the fields its writer thought to name. A library change takes the version bump alone, since a library is rebuilt. -### Backfilling a release that predates the command +**The constant already stands ahead of the release.** The pending step is the one to widen. Fold the new transform into the module named after that version, describe the whole step from the shipped version in its docstring, and extend its tests to cover what was added. The version constant stays where it is, because the pending step already carries users from the shipped version. -Run the writer inside a worktree at that tag, keeping the port beside the files it -wrote: +The engine stamps the new version once the chain runs, so a step module declares only its own transform. -``` -git worktree add / -cd / && uv sync --frozen -cp /tests/data/compatibility/generators/.py . -uv run python .py --output /tests/data/compatibility -cd && git worktree remove --force / -``` +## The corpus -The sync runs in the worktree rather than against the current environment because -a stored document names the release that wrote it, which the package's own -installed metadata answers for. +A step tested against a payload the test builds itself is held only to the fields whoever wrote the test thought to name. `tests/data/compatibility` therefore keeps one stored document per format per shipped data version, written by the build that shipped it. `tests/integration/compatibility` opens each one through that format's own load entry point and holds the loaded model to the current shape. -## Verification +Two of those tests hold the corpus itself together. A file has the version its name says, and every registered step reads a version the corpus keeps. The second makes opening a step and archiving the file it reads one act and not two. -- `uv run pytest tests/unit/sampletones_core/compatibility` covers the engine and - every registered step against payloads built for it. -- `uv run pytest tests/integration/compatibility` opens the archived documents and - holds them to the shape this build reads. +The corpus states its own rules, and how a version is written into it, in `tests/data/compatibility/README.md`. diff --git a/docs/development/release/dependencies.md b/docs/development/release/dependencies.md index b0c0fbd0b..3270fe166 100644 --- a/docs/development/release/dependencies.md +++ b/docs/development/release/dependencies.md @@ -1,152 +1,69 @@ # Dependencies +This document says why each dependency exists and what it needs from the machine. Consult it when adding a dependency, changing the build, or diagnosing an install that fails. `pyproject.toml` states every Python dependency and the version each is held to. This page covers what `pyproject.toml` cannot say: system libraries, build-time tools and the reason each one is there. + +Dependencies fall into groups by who needs them: what ships with the application, what only the build needs, and what only developers need. Each section below says which group it belongs to. + ## Graphical interface -The graphical user interface is implemented with DearPyGui, a Python wrapper for ImGui (https://www.dearimgui.com/). +The interface is built with [DearPyGui](https://github.com/hoffstadt/DearPyGui), a Python wrapper for [Dear ImGui](https://www.dearimgui.com/). It ships with the application. ## Core -The core depends on common Python packages: -* `numpy` -* `scipy` -* `librosa` -* `cupy` (optional; enables the GPU backend, with the build selected for your NVIDIA driver) - -See [GPU acceleration](../../guide/installation.md#gpu-acceleration) for enabling it. +The reconstruction engine stands on the usual numerical stack, with `cupy` as an optional GPU backend. The GPU backend is installed as the extra that matches the machine's NVIDIA driver. See [GPU acceleration](../../guide/installation.md#gpu-acceleration) for enabling it. ## Serialization -Instruction libraries and reconstructions are serialized with [MessagePack](https://msgpack.org/) (the `msgpack` package). No external compiler or system dependency is required — it is installed automatically with the package. +Instruction libraries and reconstructions are serialized with [MessagePack](https://msgpack.org/) (the `msgpack` package). It needs no external compiler or system dependency and installs automatically with the package. ## Audio playback -Playback goes through PortAudio, reached with the `pyaudio` package. PyPI carries `pyaudio` wheels for Windows, so Linux and macOS compile it on install and need the PortAudio headers and library on the machine. `scripts/system_dependencies.py` installs them: the distribution packages through apt on Linux, PortAudio through Homebrew on macOS. +Playback goes through PortAudio, reached with the `pyaudio` package. PyPI has `pyaudio` wheels for Windows only, so Linux and macOS compile it on install and need the PortAudio headers and library on the machine. `scripts/system_dependencies.py` installs them: the distribution packages through apt on Linux, and PortAudio through Homebrew on macOS. -Compiling on macOS also depends on the interpreter's architecture. The python.org installer ships a universal2 build, which compiles extensions for both Apple Silicon and Intel, while Homebrew's `libportaudio` carries the machine's own architecture. Pinning `ARCHFLAGS` to `uname -m` settles it on the native one: `make setup` sets it directly, and the CI workflows take it from `scripts/build_environment.py`, which reports it as a `KEY=VALUE` line alongside the PortAudio prefix for a Homebrew installed outside its usual place. +On macOS the compile architecture is pinned to the machine's own (`ARCHFLAGS`). Homebrew's PortAudio is native to the machine, while the python.org interpreter is universal2, and pinning makes the two agree. `make setup` sets it, and the CI workflows take it from `scripts/build_environment.py`. ## Audio rendering -Audio files are written with libsndfile, reached with the `soundfile` package. Its wheels carry a -prebuilt libsndfile 1.2.2 for every supported platform, so the encoders come with the package and -need nothing installed alongside them. - -Which formats an installation writes is asked of the library at runtime, because libsndfile is built -with a codec set that varies by platform and packaging — the MP3 encoder in particular arrived in -1.2.0 and is present where it was compiled in. The chooser offers the formats the library reports, -so what a user is shown describes the machine it is running on. +Audio files are written with libsndfile, reached with the `soundfile` package. Its wheels carry a prebuilt libsndfile for every supported platform, so the encoders come with the package and need nothing installed alongside them. -| Format | Sample rates | Quality | -| --- | --- | --- | -| WAV | 8000, 16000, 22050, 44100, 48000, 96000, 192000 Hz | 8, 16, 24 or 32-bit PCM, or 32-bit float | -| MP3 | 8000, 16000, 22050, 44100, 48000 Hz | a bitrate from the ladder its MPEG version defines | +The application asks the library at runtime which formats it can write, because the codec set varies by platform and packaging. The MP3 encoder, for example, is present only where libsndfile was built with it. The chooser therefore offers the formats the library reports, so what a user is shown describes the machine it runs on. -The bitrates on offer narrow with the sample rate: up to 320 kbps at 44100 and 48000 Hz, 160 kbps at -16000 and 22050 Hz, and 64 kbps at 8000 Hz. libsndfile takes MP3 quality as a compression level -between 0 and 1 and turns it into a rung on that ladder, so a bitrate is reached through the level -its rate maps it to, measured per rate and held in `sampletones_core/audio/writers/bitrate.py`. +The writers (`sampletones_core/audio/writers/`) say which sample rates and qualities each format offers. libsndfile takes MP3 quality as a compression level and not a bitrate, and the bitrates a rate can carry narrow as the rate falls. `writers/bitrate.py` therefore maps a level to the bitrate it reaches at each rate. ## File dialogs -Dialogs open through the XDG desktop portal (`org.freedesktop.portal.FileChooser`), reached over D-Bus with the pure-Python `jeepney` package on Linux. The portal lists every offered file type in its selector and reports back the one the user picked, which is what lets a save settle its format from the type chosen there. Where no portal answers, `kdialog` and `zenity` take over, and Tk last. +Dialogs open through the XDG desktop portal (`org.freedesktop.portal.FileChooser`), reached over D-Bus with the pure-Python `jeepney` package on Linux. The portal lists every offered file type in its selector and reports back the one the user picked, so a save settles its format from the type chosen there. Where no portal answers, `kdialog` and `zenity` take over, and Tk last. -`jeepney` is declared for Linux alone, so the modules that speak to the portal are imported where it is installed: the application probes for it before reaching them, and the root `conftest.py` keeps them out of collection elsewhere, leaving the Linux runs of the suite to cover them. +`jeepney` is declared for Linux only, so the modules that speak to the portal are imported only where it is installed. The application probes for it before reaching them, and the root `conftest.py` keeps them out of collection elsewhere, so the Linux runs of the suite cover them. ## Application icon -The icon suite in `src/sampletones_assets/icons` is generated from the mark declared in -`src/sampletones_tools/assets/mark/config`: `mark.yaml` carries the geometry, colors and rasterization -settings, validated as a `Mark`, and `template.svg` is the vector the rendered geometry fills. The -tools package writes the whole suite — the vector `sampletones.svg` and the rasters the application -ships, `sampletones.png` and the multi-resolution `sampletones.ico` — and `uv run sampletones icons` -points it at the directory the icons are shipped from. Rasterization uses Pillow, declared in the -`assets` dependency group, which the `dev` group includes. - -The whole suite is committed, so a plain checkout carries the icons the application opens its window -with, and every wheel, bundle and test run finds them where they lie. `uv run sampletones icons` -writes them again from the mark, and the `icons` pre-push hook writes them for a push that touches -either directory, holding the committed files to what the mark describes. CI runs that same hook. - -Pillow is a developer tool the build environment never installs, and the bundle script passes -`--exclude-module PIL` besides, to hold it to that: -`pygments`, which arrives with `rich`, offers an image formatter that imports Pillow where it is -installed, and PyInstaller follows that import into the bundle. The application reads its icons as -files, so the exclusion spares every bundle Pillow's extension modules and the imaging libraries -that come with them. `scripts/verify_bundle.py` holds the release bundles to it. +The icon suite is generated from a mark declared in `sampletones_tools/assets/mark/config` and committed, so a plain checkout has the icons the application opens its window with. [Tooling](../tooling.md) describes the `icons` command that writes it. + +Pillow rasterizes the suite. It is declared in the `assets` dependency group, which the `dev` group includes. It is a developer tool that the build environment never installs, and the bundle script passes `--exclude-module PIL` as well. `pygments`, which arrives with `rich`, offers an image formatter that imports Pillow where it is installed, and PyInstaller follows that import into the bundle. The application reads its icons as files, so the exclusion keeps Pillow's extension modules and the imaging libraries that come with them out of every bundle. `scripts/verify_bundle.py` holds the release bundles to it. ## Calibration -The calibration harness scores renders with referees of its own, built on `numpy` and `scipy`. The -`calibration` dependency group adds [Zimtohrli](https://github.com/google/zimtohrli), a -psychoacoustic model, as a second opinion. PyPI carries its wheels for Windows, Intel macOS and -x86-64 Linux, and other systems compile it on install. The group stays out of `dev`: -`uv sync --group calibration`, with the extras the environment already uses named beside it, -installs it. See [Calibration](../../tools/calibration.md). +The calibration harness scores renders with referees of its own, built on `numpy` and `scipy`. The `calibration` dependency group adds [Zimtohrli](https://github.com/google/zimtohrli), a psychoacoustic model, as a second opinion. PyPI has its wheels for Windows, Intel macOS and x86-64 Linux, and other systems compile it on install. The group stays out of `dev`. `uv sync --group calibration`, with the extras the environment already uses named beside it, installs it. See [Calibration](../../tools/calibration.md). ## NES player driver -The player that runs on the console is 6502 assembly, held in three parts: -`src/sampletones_tools/player/assembly/` carries the sources, their includes and the linker -configuration, `src/sampletones_tools/player/assembler/` carries the Python that assembles them, -and `src/sampletones_player/driver/binary/` carries the assembled `driver.bin` the application -ships. `uv run sampletones driver` runs the build, so it behaves the same on every system the -project supports. - -Assembling needs `ca65` and `ld65` from [cc65](https://cc65.github.io/) — on Debian and Ubuntu, -`sudo apt install cc65`, and a build names the equivalent for whichever system it runs on when the -programs are absent. cc65 is a build-time tool for the driver alone. - -The assembled `driver.bin` is committed, so a checkout carries the player and exporting an NSF -needs no assembler. A jump table leads the image, which fixes the addresses an NSF header names -whatever the driver's length, so the exporter states them from `specification/driver.py` and a -build holds the linker's own labels to them before it writes anything. Editing the assembly means -running `uv run sampletones driver` again and committing what it writes; the driver's test suite rebuilds the -sources and holds the committed image to them wherever cc65 is installed. The wheel carries the -assembled image, which is what exporting reads, and the assembly sources inside the tools package, -so `sampletones driver -o DIR` assembles the driver from an installed copy too. - -cc65 is distributed under the zlib license, and the driver stays clear of it: the link line names -our own object files and our own `nsf.cfg`, so nothing of cc65's start-up code or libraries reaches -the committed image. That keeps the blob entirely ours to ship under the project's MIT license. +Assembling the console player needs `ca65` and `ld65` from [cc65](https://cc65.github.io/). On Debian and Ubuntu that is `sudo apt install cc65`, and a build names the equivalent for whichever system it runs on when the programs are absent. cc65 is a build-time tool for the driver alone. The assembled `driver.bin` is committed, so a checkout has the player and exporting an NSF needs no assembler. Editing the assembly means running `uv run sampletones driver` again and committing what it writes. + +cc65 is distributed under the zlib license. The link line names our own object files and our own `nsf.cfg`, so nothing of cc65's start-up code or libraries reaches the committed image. That keeps the blob entirely ours to ship under the project's MIT license. ### Verifying the driver -`tests/integration/nsf` runs an exported file the way a console runs it. [py65](https://github.com/mnaberez/py65) -— a 6502 emulator in the `dev` dependency group — executes the assembled driver against memory that -watches the APU's address range, so each routine answers with the register writes it made and the -suite holds the whole run against `RegisterTrace.from_song`. Reading those writes back into -instructions and rendering them through the project's own generators closes the loop on the sound -as well: what the console plays stands against the very waveform the reconstruction carries. py65 -is a developer dependency, outside both the wheel and the bundles, and its BSD license leaves the -project's own terms untouched. - -Listening to a real APU needs [ffmpeg](https://ffmpeg.org/) carrying the `libgme` demuxer, which -is a build option rather than a given: `uv run sampletones nsf render` asks the installed ffmpeg which demuxers -it holds and names this system's install command before it decodes anything. `nsf samples -o DIR` -writes the example files, and `nsf render --directory DIR` renders each one to a wave beside it, its -length read out of the song block the file carries. That is an ear rather than a gate: the register trace is what the driver answers to, and -the wave is what a person listens to. - -### The player's tools - -Three tools serve the player, each reached by one command: - -| Tool | Run by | Installed with | Reaches | -| --- | --- | --- | --- | -| cc65 (`ca65`, `ld65`) | `sampletones driver` | the system's package manager | the machine assembling the driver | -| py65 | `make test` | `uv sync --group dev` | the `dev` dependency group | -| ffmpeg with `libgme` | `sampletones nsf render` | the system's package manager | the machine listening to an export | - -`scripts/system_dependencies.py` carries what building and running the application needs, and the -workflows install the `dev` group, so py65 is the one of the three CI -reaches — the suite verifies the driver through it alone. cc65 and ffmpeg stay on the machine of -whoever runs `sampletones driver` or `sampletones nsf render`, and a workflow that assembles the driver or renders -a wave is what would put them in those scripts. The application itself calls neither: an export is -written by the package's own code, from the committed `driver.bin`. +[py65](https://github.com/mnaberez/py65), a 6502 emulator in the `dev` dependency group, executes the assembled driver against memory that watches the APU's address range. That lets the suite hold the image to what a correct driver writes ([the console player](../player.md)). py65 is a developer dependency, outside both the wheel and the bundles, and its BSD license leaves the project's own terms untouched. + +Listening to a real APU needs [ffmpeg](https://ffmpeg.org/) with the `libgme` demuxer, which is a build option and not a given. `uv run sampletones nsf render` asks the installed ffmpeg which demuxers it has, and names this system's install command before it decodes anything. + +CI reaches only py65, because the workflows install the `dev` group and `scripts/system_dependencies.py` has what building and running the application needs. cc65 and ffmpeg stay on the machine of whoever assembles the driver or renders a wave. A workflow that did either would put them in those scripts. The application itself needs neither: an export is written by the package's own code, from the committed `driver.bin`. ## Linux (standalone executable) -Building a standalone executable on Linux needs the PortAudio, Tk and OpenGL/X11 system packages. Install them with `make system-deps` (or run `python3 scripts/system_dependencies.py`), which holds the full list. +Building a standalone executable on Linux needs the PortAudio, Tk and OpenGL/X11 system packages. `make system-deps` installs them (or run `python3 scripts/system_dependencies.py`), and the script has the full list. PortAudio is required. Tk backs the file dialogs where neither a portal nor a desktop tool answers, and `make release` requires it so the shipped executable stays self-contained. -The executable links against the glibc of the machine that builds it and runs on that version or newer, so a redistributable artifact belongs on the oldest Debian or Ubuntu release being supported. +The executable links against the glibc of the machine that builds it and runs on that version or newer. A redistributable artifact therefore belongs on the oldest Debian or Ubuntu release being supported, so it runs on every newer one. diff --git a/docs/development/tooling.md b/docs/development/tooling.md index ad654e92b..6a612e313 100644 --- a/docs/development/tooling.md +++ b/docs/development/tooling.md @@ -3,45 +3,50 @@ This document governs how the repository is run: the `sampletones` command and what it offers, the tools package behind its developer commands, the scripts under `scripts/`, and the `Makefile`. Read it before adding a command, a tool, a script or a make target. Which packages may import which is -[package layers](packages.md); the libraries and tools the scripts reach for are +[package layers](packages.md). The libraries and tools the scripts reach for are [dependencies](release/dependencies.md). ## Principles -**1. One door.** Everything a person runs by hand is a `sampletones` command with its own parser -and help, always named: `sampletones` alone is `sampletones run`, a file is opened with -`sampletones open `, a recording is converted with `sampletones convert `. A -command's name says what it does, in plain words. - -**2. Three kinds of runnable code, told apart by who runs them and what they may import.** The -*application* is what a user installs. A *tool* runs inside the project environment, through -`uv run`: it may import any package, and only the command line reaches it. A *bootstrap script* -runs on the system interpreter, before or beside the environment: it creates the environment, -installs system packages, builds the standalone bundle, cleans the tree, and runs the tests, the -linters and the formatters the environment provides. It imports the standard library and the other -bootstrap modules, nothing else, so it runs on a machine that has Python 3.12 or newer and nothing -more. Importing `scripts/bootstrap/` checks that version before anything else, so an older -interpreter is told the version and where to download it. A bootstrap script installs nothing into -the interpreter it runs on: every package a build installs lands in `.venv-build`, a virtual -environment of its own, and pip is told to refuse any interpreter outside one. System packages are a step of their own, `make system-deps`, the only one that asks for -administrator rights. The import boundary check holds the tree to the rule: -`sampletones_config/boundaries/standalone.yaml` names the scripts, and an import beyond the -standard library and the tree fails the hook. - -**3. What earns a place on the command.** An operation is a *user command* when its input and -output are the user's own files and it needs nothing beyond the installed package. It is a -*developer command* when it reads or writes the repository or measures the code on this machine, -so it needs a checkout and the project environment. It is a *bootstrap script* when it must run -without the environment. A test is run by pytest and is never wrapped in a command; a tool the -tests exercise is a function they call. - -**4. A tool is a library with a thin face.** The work is a function that takes values and returns -values; the command module parses the arguments, calls it and prints. A command module imports the -standard library, the command type and its constants at module level, and its implementation -inside `run`, so listing the commands loads no tool. A bootstrap script is shaped the same way: -pure functions assemble the commands and take the decisions, `main` wires in the real runner and -the real environment, and the tests call the functions with a runner that records what it was -asked to run, so a build is verified without building. +**1. One entry.** Everything a person runs by hand is a `sampletones` command with its own parser and +help, always named. `sampletones` alone is `sampletones run`, a file is opened with +`sampletones open `, and a recording is converted with `sampletones convert `. A command's +name says what it does, in plain words. + +**2. Runnable code comes in kinds that differ by who runs it and what it may import.** + +- The *application* is what a user installs. +- A *tool* runs inside the project environment, through `uv run`. It may import any package, and only the + command line reaches it. +- A *bootstrap script* runs on the system interpreter, before or beside the environment. It creates the + environment, installs system packages, builds the standalone bundle, cleans the tree, and runs the + tests, the linters and the formatters the environment provides. + +A bootstrap script imports the standard library and the other bootstrap modules and nothing else, so it +runs on a machine that has Python 3.12 or newer and nothing more. Importing `scripts/bootstrap/` checks +the version before anything else, so an older interpreter is told the version and where to download it. + +A bootstrap script installs nothing into the interpreter it runs on. Every package a build installs lands +in `.venv-build`, a virtual environment of its own, and pip is told to refuse any interpreter outside one. +System packages are a step of their own, `make system-deps`, and it is the only step that asks for +administrator rights. + +The import boundary check holds the tree to the rule. `sampletones_config/boundaries/standalone.yaml` names +the scripts, and an import beyond the standard library and the tree fails the hook. + +**3. What earns a place on the command.** An operation is a *user command* when its input and output are +the user's own files and it needs nothing beyond the installed package. It is a *developer command* when +it reads or writes the repository or measures the code on this machine, so it needs a checkout and the +project environment. It is a *bootstrap script* when it must run without the environment. A test is run by +pytest, and no command wraps it. A tool the tests exercise is a function they call. + +**4. A tool is a library with a thin face.** The work is a function that takes values and returns values. +The command module parses the arguments, calls the function and prints. A command module imports the +standard library, the command type and its constants at module level, and its implementation inside `run`, +so listing the commands loads no tool. A bootstrap script has the same shape. Pure functions assemble the +commands and take the decisions, and `main` wires in the real runner and the real environment. The tests +call the functions with a runner that records what it was asked to run, so a build is verified without +building. **5. The import graph is declared once and checked.** `sampletones` is the top of the graph, and the bootstrap tree is a root of its own under the standard-library rule. [Package layers](packages.md) @@ -50,158 +55,79 @@ holds the graph and its enforcement. **6. Tests mirror the tree.** `tests/unit//` mirrors `src//` and `tests/unit/scripts/` mirrors `scripts/`. Code and its tests move in one change. -**7. One script per operation, the same on every system.** What differs between systems (the -launcher's name and extension, the icon, the package manager, the compiler flags audio playback -needs) sits behind one `Platform` protocol with an implementation per system, chosen by a factory -from `platform.system()`. A script never branches on the operating system itself, and a system the -project does not build on is refused by name. The Makefile is the developer's index, one line per -target: a target names the script that does the work and passes its flag, or, for `run` and -`calibration`, the `sampletones` command it starts with no options. The two shell files at -the root, `install.sh` and `install.bat`, exist for the double-click path and call the same bundle -script. +**7. One script per operation, the same on every system.** What differs between systems sits behind one +`Platform` protocol with an implementation per system, chosen by a factory from `platform.system()`. It +covers the launcher's name and extension, the icon, the package manager, and the compiler flags audio +playback needs. A script never branches on the operating system itself, and a system the project does not +build on is refused by name. -**8. A developer command works from what it is given, in every copy of the program.** The wheel -and the bundle carry the tools package, so every developer command exists wherever `sampletones` -is installed, and each one reaches files the same way in all of them: +The Makefile is the developer's index, one line per target. A target names the script that does the work +and passes its flag. The `run` and `calibration` targets name the `sampletones` command they start with no +options. `install.sh` and `install.bat` at the root exist for the double-click path and call the same +bundle script. + +**8. A developer command works from what it is given, in every copy of the program.** The wheel and the +bundle carry the tools package, so every developer command exists wherever `sampletones` is installed, and +each one reaches files the same way in all of them: - *Inputs* arrive on the command line or in a file a run wrote, so a run starts from what the person running it has. -- *Outputs* go where `--output` (`-o`) names. A measurement given no `-o` writes a timestamped - directory under the user's Documents; `codec report` and the sample emitters take `-o` always. -- *The repository* is reached through the checkout guard. A command that reads or writes the - repository, or needs a development dependency, runs from a checkout; so `driver` and `icons` - rewrite the files the package ships from a checkout, and a measurement runs anywhere. -- *Package data* is read from the package it ships in, which holds in a checkout, in the wheel and - in the bundle. +- *Outputs* go where `--output` (`-o`) names. A measurement given no `-o` writes a timestamped directory + under the user's Documents. `codec report` and the sample emitters always take `-o`. +- *The repository* is reached through the checkout guard. A command that reads or writes the repository, + or needs a development dependency, runs from a checkout. `driver` and `icons` rewrite the files the + package ships from a checkout, and a measurement runs anywhere. +- *Package data* is read from the package it ships in, which holds in a checkout, in the wheel and in the + bundle. ## The commands -`src/sampletones/` is the entry package. `dispatcher.py` builds one parser over the commands and -runs the one named; `commands/` holds one module per command and `commands/registry.py` lists -them. A command is a frozen `Command` (`sampletones_shared/command.py`): its name, one line of -help, the function adding its options to a parser, and the function running it over the parsed -arguments. Each command turns its arguments into a frozen record, field by field, before it works; a -command with actions, such as `codec`, reads the action first and builds the record that action takes. -A command writing files names where they go `--output` (`-o`): a file for `convert`, a directory for -the others. An option naming an input says what it reads, so `--config` is a configuration file -wherever it appears. - -| Command | What it does | -|---|---| -| `run [--config FILE]` | Starts the application | -| `open PATH [--config FILE]` | Starts the application with a `.stp` project, a `.stn` reconstruction or an `.ins` library loaded; a recording is refused with the `convert` line to run instead | -| `convert SOURCE... [-o FILE] [--config FILE] [--channels LIST \| --stems FILE]` | Reconstructs recordings into one `.stn` file, or every recording under one directory file by file. `--stems` names a JSON file holding the setup the `.stn` record stores, its entries paired with the sources in order; the pairing is printed before the run, and a missing source, a file other than a recording or a missing stems file is refused first | -| `library [--config FILE]` | Generates the instruction library for a configuration | -| `self-check` | Verifies that the build's imports, bundled resources and configuration files are usable | - -`--version` and `--help` are flags of the entry itself. The headless runs behind `convert` and -`library` live in `sampletones_core/headless/`, where the calibration reuses them. - -The developer commands, listed by `sampletones_tools/registry.py` and run as -`uv run sampletones ` from a checkout: - -| Command | What it does | -|---|---| -| `calibration [--config FILE] [-o DIR] [--methods LIST] [--perceptual-exponents LIST] [--temporal-weights LIST] [--channels LIST] [--palette NAME] [--no-open]` | Measures how the program reconstructs the reference sounds under the packaged suite, or the parts of it the options replace, and writes the renders, the report and the page the renders are heard on; `make calibration` runs it with no options; without `-o` the run lands in a timestamped directory under Documents/SampleToNES/calibration | -| `calibration --board RUN [RUN ...] [-o DIR] [--palette NAME] [--no-open]` | Builds one listening page over finished runs, measuring nothing; without `-o` the page lands in a timestamped directory under Documents/SampleToNES/calibration/pages. [Calibration](../tools/calibration.md) explains every use | -| `check [options]` | Holds the tree to one of its checks: `import-boundary`, `language-keys`, `palette-colors`, `rendered-literals`, `shortcut-actions`, `tag-names`, `unused-tags`; each is a pre-commit hook, and [architecture](architecture.md#enforcement) says what each holds. Needs a checkout | -| `btp samples -o DIR`, `ftm samples -o DIR`, `nsf samples -o DIR` | Builds the synthetic corpus and writes it as example files: the arrangement as two Bitphase documents (at its tempo and as a groove), the arrangement as a FamiTracker module, or each sample and the arrangement as `.nsf` programs | -| `compatibility [-o DIR] [--force]` | Archives one document per stored format at the data versions this build writes, into `tests/data/compatibility`, so a later build can open what this one wrote; a version already archived stands as it was written unless `--force` replaces it. Needs a checkout | -| `codec report -o DIR` | Compresses the synthetic corpus under every layer of the codec and writes the report the format's constants are settled from, as CSV and Markdown | -| `codec study [--manifest FILE] [--project FILE]... [--reconstruction PATH]... [-o DIR] [--lengthen SECONDS] [--variants LIST]` | Encodes the projects and stems it is given, or the ones a manifest names, under every candidate change to the codec and writes the sizes, the times, a verdict per candidate and the manifest that repeats the run; without `-o` the run lands under Documents/SampleToNES/compression | -| `driver [-o DIR]` | Assembles the NES player driver with cc65 and prints the layout the build produced; without `-o` it writes the driver the package ships, which needs a checkout | -| `icons [-o DIR]` | Writes the icon suite from the mark, into `-o` or over the icons the package ships. Needs a checkout | -| `nsf render --directory DIR [--tail SECONDS]` | Renders every exported `.nsf` file in the directory to a wave beside it, through ffmpeg's libgme demuxer | +`src/sampletones/` is the entry package. Its dispatcher builds one parser over the commands and runs the +one named. The registries under `commands/` and `sampletones_tools/` list the user commands and the +developer commands. A command is a frozen `Command`: its name, one line of help, the function that adds its +options to a parser, and the function that runs it over the parsed arguments. `--version` and `--help` are +flags of the entry itself, and `sampletones --help` lists what a build offers. + +These conventions hold the command surface together: + +- Each command turns its arguments into a frozen record, field by field, before it works. +- A command with actions, such as `codec`, reads the action first and builds the record that action takes. +- A command that writes files names where they go `--output` (`-o`). +- An option that names an input says what it reads, so `--config` is a configuration file wherever it + appears. +- A refused value is reported on a line of its own, through `describe_failure`. + +The headless runs behind `convert` and `library` live in `sampletones_core/headless/`, where the +calibration reuses them. ## The tools package -`src/sampletones_tools/` holds every tool the running application does not use, in subpackages by -subject, and `sampletones_tools/registry.py` lists the developer commands they offer. -`sampletones/commands/registry.py` appends them to the user commands, which is the one import of -the tools package; [package layers](packages.md) holds the edge. Two helpers carry out principle 8: - -- `sampletones_tools/checkout.py` holds `require_checkout(command)`: the repository root holds - `pyproject.toml` beside `src/`, or the command exits naming `uv run sampletones ` in a - checkout. `icons` calls it for Pillow, a development dependency, as well as for the repository. -- `package_directory` in `sampletones_shared/paths/package.py` places a package from the import - system's own record, where PyInstaller unpacks each package's data beside its modules. - -A tool that writes a page ships that page's files as they are read. `calibration/board/static/` -holds the markup, the stylesheet and the script, copied out byte for byte, and the builder writes -the palette, the faces and the run's own contents beside them. The stylesheet names color tokens -and the script names no measurement, so a page is edited in place and drawn in the application's -palettes; a test holds every shipped file to reaching nothing beyond the page, which is what lets -one open from a file. - -Developer commands are run as `uv run sampletones ` from a checkout; the `sampletones` -command `make setup` installs is a wheel and refuses the guarded ones the same way. A command module -imports pillow, NumPy and the like inside `run`, and a test imports the registry in a subprocess and -asserts that only the command, registry and package modules of the tools load and no heavy library -does, since a startup failure in any tool module would break every invocation, the GUI included. A -command reports each refused value on a line of its own: `describe_failure` in -`sampletones_shared/utils/validation.py` renders a validation error the way a person reads it. The -editable install puts `src/` on the path whole, so a checkout -sees the tools package whatever the wheel lists; hatchling's `dev-mode-exact` stays off for that -reason. +`src/sampletones_tools/` holds every tool the running application does not use, in subpackages by subject. +Two helpers carry out principle 8: + +- `sampletones_tools/checkout.py` holds `require_checkout(command)`. It passes when the repository root has + `pyproject.toml` beside `src/`, and otherwise the command exits and names `uv run sampletones ` + in a checkout. `icons` calls it for Pillow, a development dependency, as well as for the repository. +- `package_directory` in `sampletones_shared/paths/package.py` places a package from the import system's + own record, where PyInstaller unpacks each package's data beside its modules. + +A test guards principle 4. It imports the registry in a subprocess and asserts that only the command, +registry and package modules of the tools load and no heavy library does. A startup failure in any tool +module would break every invocation, the GUI included. + +The `icons` command writes the icon suite from the mark declared in `sampletones_tools/assets/mark/config`. `mark.yaml` has the geometry, colors and rasterization settings, validated as a `Mark`, and `template.svg` is the vector the rendered geometry fills. The suite is the vector `sampletones.svg` and the rasters the application ships, `sampletones.png` and the multi-resolution `sampletones.ico`. The command points at the directory the icons ship from. The whole suite is committed, so every wheel, bundle and test run finds the icons where they lie. The `icons` pre-push hook writes them again for a push that touches either directory, which holds the committed files to what the mark describes, and CI runs that same hook. + +A tool that writes a page ships that page's files as they are read. `calibration/board/static/` holds the +markup, the stylesheet and the script, copied out byte for byte, and the builder writes the palette, the +faces and the run's own contents beside them. The stylesheet names color tokens and the script names no +measurement, so a page is edited in place and drawn in the application's palettes. A test holds every +shipped file to reaching nothing beyond the page, which lets one open from a file. ## The bootstrap scripts -| Script | Target | What it does | -|---|---|---| -| `bundle.py` | `make build`, `make release` | Creates `.venv-build`, installs the package with the `build` extra, checks the interpreter carries PortAudio (and Tk, for a release), writes the bundle with PyInstaller, runs its self-check, and copies the notices beside a release. Every package the wheel carries brings its data files at its own package path, so the frozen application finds them where an installed one does | -| `setup_environment.py` | `make setup` | Reads the NVIDIA driver, synchronizes the development environment with the matching GPU extra, installs the global `sampletones` command | -| `system_dependencies.py` | `make system-deps` | Installs the system packages: apt on Debian-based Linux, Homebrew on macOS, nothing on Windows | -| `build_environment.py` | CI | Prints the compiler flags a macOS build exports, one `KEY=VALUE` per line | -| `clean.py` | `make clean` | Removes the build outputs, the coverage reports and the bytecode caches | -| `run_tests.py` | `make test`, `make test-docs`, `make benchmarks` | Runs one pass of the tests, named on its command line: `suite`, the covered suite across six workers (`--workers` sets the count); `doctests`; or `benchmarks`, serial and uncovered. Each pass is a target, a pre-push hook and a CI step of its own, so a failure names its pass | -| `lint.py` | `make lint` | Runs mypy over the files `pyproject.toml` configures and pylint over `src/` and `scripts/`; `--mypy` or `--pylint` picks one, and named paths narrow both | -| `formatting.py` | `make format` | Runs isort, then black, over `src/`, `tests/` and `scripts/`, or over the paths named | -| `hooks.py` | `make pre-commit` | Installs the git hooks pre-commit runs at commit and at push | -| `runtime_hooks/release_environment.py` | build input | The PyInstaller runtime hook that gives a release bundle its deployment defaults | -| `verify_version_tag.py` | the release workflow | Holds the release tag to the version `pyproject.toml` records | -| `verify_bundle.py` | the release workflow | Holds the release bundle to its notices, keeps the build tools out of it, and starts its launcher | -| `archive_bundle.py` | the release workflow | Zips the release bundle into `bundles/`, named by the version and the `--label` of the platform | - -`scripts/bootstrap/` holds what they share, one fact in one place: - -- `layout.py`: the repository root and every path and list a script names: `bin`, `bundles`, - `.venv-build`, the runtime hook, the notices, the build tools a bundle leaves out, and what - `make clean` removes. -- `project.py`: what `pyproject.toml` states, read with `tomllib`: the name, the version, the - entry module, the wheel's packages, and the extras and groups the scripts install, which it - holds the file to. -- `platforms/`: what differs between systems, behind `Platform`, with `Bundling` holding what a - bundle takes on a system that builds one. -- `cuda.py`: the NVIDIA driver's CUDA version and the CuPy extra it selects. -- `interpreter.py`, `processes.py`, `passes.py`, `files.py`: the interpreter version check, - running a command and holding it to success, a run of named passes that reports every failure - at once, and removing a file or a tree. -- `venv_build.py`, `preflight.py`: the build environment and the installs into it, and the - preflight of the build interpreter. - -Every script's work is a function taking what it reads: the repository root, the platform, and for -a script running commands the runner and the variables. `main` parses the arguments and passes in -the real ones, and the tests pass a temporary repository and a `RecordingRunner`. - -## Who governs what - -| Concern | Owner | -|---|---| -| Which commands the entry offers | `src/sampletones/commands/registry.py` | -| Which developer commands exist | `src/sampletones_tools/registry.py` | -| Which checks the tree is held to | `src/sampletones_tools/checks/registry.py` | -| Whether a command runs outside a checkout | `src/sampletones_tools/checkout.py` | -| The synthetic corpus the emitters and the integration tests share | `src/sampletones_tools/corpus/` | -| The page a calibration run is listened to on | `src/sampletones_tools/calibration/board/` | -| What a command is | `src/sampletones_shared/command.py` | -| What a bootstrap script may import | `sampletones_config/boundaries/standalone.yaml` | -| What differs between systems | `scripts/bootstrap/platforms/` | -| Where a script finds a path, a notice or a clean target | `scripts/bootstrap/layout.py` | -| What the scripts read from `pyproject.toml` | `scripts/bootstrap/project.py` | -| Where a build installs | `scripts/bootstrap/venv_build.py` | -| What a bundle has to carry before it is built | `scripts/bootstrap/preflight.py` | -| The PyInstaller invocation | `scripts/bundle.py` | -| The test passes and the command each runs | `scripts/run_tests.py` | -| What `make lint` and `make format` sweep | `scripts/lint.py`, `scripts/formatting.py` | -| The GPU extra a machine gets | `scripts/bootstrap/cuda.py` | -| What a release is held to | `scripts/verify_version_tag.py`, `scripts/verify_bundle.py` | +`scripts/` holds one script per operation (principle 7). What they share sits in `scripts/bootstrap/`, one +fact in one place. `layout.py` holds the repository root and every path, notice and clean target a script +names. `project.py` holds what `pyproject.toml` says, read with `tomllib`, which it holds the file to. + +The tests are run as named passes: the suite, the doctests and the benchmarks. Each pass is a make target, +a pre-push hook and a CI step of its own, so a failure names its pass. diff --git a/docs/formats/bitphase.md b/docs/formats/bitphase.md index 7a6ddc57a..21c146c68 100644 --- a/docs/formats/bitphase.md +++ b/docs/formats/bitphase.md @@ -1,34 +1,30 @@ # Bitphase export format -This document is the reference for how _SampleToNES_ writes -[Bitphase](https://github.com/paator/bitphase) files. It describes the two files the -`sampletones_core.formats.bitphase` package produces — the `.btp` document and the -`.json` instrument preset — and the Bitphase capacity limits the exporter respects. -Read it before changing anything under `formats/bitphase/`; the sibling +This document is the reference for how _SampleToNES_ writes [Bitphase](https://github.com/paator/bitphase) +files: the `.btp` document and the `.json` instrument preset. It also lists the Bitphase capacity limits +the exporter respects. Read it before changing anything under `formats/bitphase/`. The sibling [FamiTracker export](famitracker.md) document covers the other tracker. -The target is Bitphase's **NES (2A03) chip**: five channels (two squares, triangle, -noise, DPCM), with the DPCM channel always silent by design. Every constant referenced -here has a named counterpart under `sampletones_core/formats/bitphase/specification/` -(grouped by unit: `chip`, `channels`, `instruments`, `patterns`). +The target is Bitphase's **NES (2A03) chip**: five channels (two squares, triangle, noise, DPCM). The DPCM +channel is always silent. Every constant named here has a counterpart under +`sampletones_core/formats/bitphase/specification/`, grouped by unit (`chip`, `channels`, `instruments`, +`patterns`). -Bitphase plays a note by three columns acting together, and that shapes the whole -mapping: an **instrument** supplies the per-tick register values, a **table** supplies -the per-tick pitch movement, and the **note column** supplies the pitch they move -around. A reconstruction's volume and duty envelopes become the instrument, its -arpeggio envelope becomes the table, and its reference pitch becomes the note. +Bitphase plays a note with three columns acting together. An **instrument** supplies the per-tick register +values. A **table** supplies the per-tick pitch movement. The **note column** supplies the pitch they move +around. This shapes the whole mapping: a reconstruction's volume and duty envelopes become the instrument, +its arpeggio envelope becomes the table, and its reference pitch becomes the note. ## A. File formats ### A.1 `.btp` — the document -A `.btp` is the document's JSON under gzip — no header and no version field. The -exporter writes it without separator padding and with a fixed gzip timestamp, so -exporting an unchanged document twice yields identical bytes. Written by -`formats/bitphase/btp.py`. +A `.btp` is the document's JSON under gzip, with no header and no version field. The exporter writes it +without separator padding and with a fixed gzip timestamp, so exporting an unchanged document twice yields +identical bytes. -Bitphase's loader reads each field on its own and falls back to a default for any it -misses, so a document that carries every field below loads exactly as it was written. +Bitphase's loader reads each field on its own and falls back to a default for any it misses. A document +with every field below loads exactly as it was written. ``` Project { name, author, songs[], loopPointId, patternOrder[], tables[], @@ -42,29 +38,24 @@ Table { id, rows[], loop, name } Instrument { id, chipType, rows[], loop, name } ``` -Instruments and tables belong to the **project** rather than to a song, so every song -addresses the same lists. `patternOrder` names the pattern each order position plays, -and `loopPointId` is the order position playback returns to. +The project owns the instruments and tables, so every song addresses the same lists. `patternOrder` names +the pattern each order position plays. `loopPointId` is the order position playback returns to. -**Field names are camelCase.** The Pydantic models under `formats/bitphase/model/` -carry snake_case attributes and serialize through a camelCase alias generator, so the -Python side reads like the rest of the codebase while the file reads like Bitphase's. +**Field names are camelCase**, as in Bitphase's own files. ### A.2 `.json` — the instrument preset -Bitphase's instruments panel saves and loads a single instrument at runtime through a -file picker. The file holds `{ chipType, name, loop, rows }`, indented the way Bitphase -writes its own, so a preset written here reads like one saved from the tracker. Written -by `formats/bitphase/preset.py`. +Bitphase's instruments panel saves and loads a single instrument through a file picker. The file holds +`{ chipType, name, loop, rows }`, indented the way Bitphase writes its own, so a preset written here reads +like one saved from the tracker. -A preset carries rows alone, so its pitch movement rides in each row's `toneAdd` -(section C.3) rather than in a table. +A preset has rows only, so its pitch movement goes in each row's `toneAdd` (section C.3) instead of a +table. ## B. The NES instrument -An instrument advances **one row per engine tick** while a note sounds, so a row -carries every register value the channel takes for that tick. From -`formats/bitphase/model/instrument.py`, matching Bitphase's `NesInstrumentRow`: +An instrument advances **one row per engine tick** while a note sounds, so a row has every register value +the channel takes for that tick. The fields match Bitphase's `NesInstrumentRow`: | Field | Range | Runtime meaning | What the exporter writes | | --- | --- | --- | --- | @@ -77,106 +68,96 @@ carries every register value the channel takes for that tick. From | `retrigger` | bool | restarts the waveform phase this tick | `false`, so the waveform runs continuously | | `sweep` / `sweepRate` / `sweepShift` | bool / 0–7 / −7–7 | the square channel's hardware sweep | disabled | -**Looping.** Playback returns to the instrument's `loop` row once it runs off the end, -which is the only mode there is. Bitphase reads every dimension out of one row, so the -instrument returns to the earliest row any dimension repeats from and each dimension goes -on sounding what it would have sounded. A slice whose dimensions all halt sets -`loop = len - 1` and rests on the level that row carries — silence where the volume -envelope ends on a note-off item, the channel's own level where the slice holds its -volume. - -**A hand-written instrument's slices.** Bitphase bakes a channel's registers tick by -tick, so an [instrument](../glossary.md#instrument) written by hand reaches a document -as a slice per channel it sounds on, each reading the dimensions that channel offers -and moving around the pitch it states. The envelopes are one set whatever the channel, -so the slices differ only in what each channel reads of them. - -**A held volume.** A slice whose volume envelope carries no item leaves its level to the -channel, so the exporter writes a full `volumeOrRate` for every frame the slice -describes. Playback combines a row's level with the pattern's volume column through a -PT3 volume table, where a full-level row comes out at the column's own level, so those -rows sound at whatever level the channel carries — the same reading FamiTracker gives a -disabled volume sequence. A slice describing no frame at all is what writes a single -silent row, the smallest instrument Bitphase plays. - -**Equal lengths.** Instrument rows and table rows advance on independent per-tick -counters, so they share a length and a loop point and stay in step for as long as the -note sounds. The slice's longest dimension supplies that shared length, and every -shorter one holds the value it ended on for the rest of it (`Envelope.resized`), which -is what the sequences of a FamiTracker instrument each do on a counter of their own. +**Looping.** Playback returns to the instrument's `loop` row once it runs off the end. That is the only +mode. Bitphase reads every dimension out of one row, so the instrument returns to the earliest row any +dimension repeats from, and each dimension goes on sounding what it would have sounded. A slice whose +dimensions all halt sets `loop = len - 1` and rests on the level that row has: silence where the volume +envelope ends on a note-off item, the channel's own level where the slice holds its volume. + +**A hand-written instrument's slices.** Bitphase bakes a channel's registers tick by tick. An +[instrument](../glossary.md#instrument) written by hand therefore reaches a document as one slice per +channel it sounds on. Each slice reads the dimensions that channel offers and moves around the pitch the +instrument states. The envelopes are one set for every channel, so the slices differ only in what each +channel reads of them. + +**A held volume.** A slice whose volume envelope has no item leaves its level to the channel. The exporter +writes a full `volumeOrRate` for every frame the slice describes. Playback combines a row's level with the +pattern's volume column through a PT3 volume table, where a full-level row comes out at the column's own +level. Those rows therefore sound at whatever level the channel has, which is how FamiTracker reads a +disabled volume sequence. A slice that describes no frame at all writes a single silent row, the smallest +instrument Bitphase plays. + +**Equal lengths.** Instrument rows and table rows advance on independent per-tick counters, so they share a +length and a loop point to stay in step for as long as the note sounds. The slice's longest dimension sets +that length. Every shorter dimension holds the value it ended on for the rest of it (`Envelope.resized`), +as the sequences of a FamiTracker instrument each do on a counter of their own. ## C. Pitch ### C.1 The tuning table -A song carries a 96-entry `tuningTable`, one channel period per note index, built by -`formats/bitphase/tuning.py` as a port of Bitphase's `generate12TETTuningTable`: +A song has a 96-entry `tuningTable`, one channel period per note index. It is a port of Bitphase's +`generate12TETTuningTable`: ``` frequency = a4TuningHz * 2 ^ ((index - 45) / 12) period = round(chipFrequency / 16 / frequency) clamped to 1..2047 ``` -Rounding matches JavaScript's `Math.round` (half away from zero on positives), so a -table built here equals the one Bitphase derives from the same settings. The exporter -writes NTSC (1 789 773 Hz) at concert pitch; PAL (1 662 607 Hz) and Dendy -(1 773 448 Hz) are named in `specification/chip.py`. +Rounding matches JavaScript's `Math.round` (half away from zero on positives), so a table built here +equals the one Bitphase derives from the same settings. The exporter writes NTSC (1 789 773 Hz) at concert +pitch. PAL (1 662 607 Hz) and Dendy (1 773 448 Hz) are named in `specification/chip.py`. -**A note index is the absolute pitch less 24**, which puts indices 0–95 over pitches -24–119 — the same span the FamiTracker exporter clamps to. A pattern cell stores that -index as a semitone and an octave, which playback resolves back with -`name - 2 + (octave - 1) * 12`. +**A note index is the absolute pitch less 24.** Indices 0–95 cover pitches 24–119, the span the FamiTracker +exporter clamps to. A pattern cell stores the index as a semitone and an octave, which playback resolves +back with `name - 2 + (octave - 1) * 12`. -The triangle channel's period is written from the same table, so a written note sounds -an octave below — the convention SampleToNES and FamiTracker already share. +The triangle channel's period comes from the same table, so a written note sounds an octave below. +_SampleToNES_ and FamiTracker share that convention. ### C.2 Tables carry the contour -A table holds one semitone offset per tick, and playback adds `rows[position]` to the -channel's note every tick. That is a direct match for a reconstruction's arpeggio -envelope in absolute mode, so the contour crosses over verbatim on the pitched -channels. +A table has one semitone offset per tick, and playback adds `rows[position]` to the channel's note every +tick. That matches a reconstruction's arpeggio envelope in absolute mode, so the contour crosses over +verbatim on the pitched channels. -A pattern's `table` column names a table by `id + 1`; `0` leaves the attached table -alone and `-1` detaches it. +A pattern's `table` column names a table by `id + 1`. `0` leaves the attached table alone and `-1` +detaches it. -**Noise** derives its period from the note index rather than from the tuning table: -playback reads `period = 15 - (index mod 16)`. Every period therefore repeats once per -sixteen indices, and the exporter picks a base index far enough below the top of the -table for a whole cycle of offsets to stay in range: +**Noise** derives its period from the note index, not from the tuning table: playback reads +`period = 15 - (index mod 16)`. Every period therefore repeats once per sixteen indices. The exporter picks +a base index far enough below the top of the table for a whole cycle of offsets to stay in range: ``` base index = 48 + ((15 - initial_period) mod 16) lands in 48..63 table offset = (-arpeggio_step) mod 16 lands in 0..15 ``` -so `15 - ((base + offset) mod 16)` is the period the reconstruction chose, wrapped into -the sixteen the channel holds. +So `15 - ((base + offset) mod 16)` is the period the reconstruction chose, wrapped into the sixteen the +channel has. ### C.3 Presets fold the contour into the period -An instrument preset carries no table, so its pitch movement is expressed as the -per-tick `toneAdd` each row applies to the note's own period. The offsets are measured -against the pitch the slice was reconstructed at, under the tuning a freshly created -Bitphase document plays — NTSC at concert pitch. The noise channel takes its period +An instrument preset has no table, so its pitch movement is the per-tick `toneAdd` each row applies to the +note's own period. The offsets are measured against the pitch the slice was reconstructed at, under the +tuning a freshly created Bitphase document plays: NTSC at concert pitch. The noise channel takes its period from the note, so its preset rows hold a flat offset. ## D. Tempo as a groove -A Bitphase song states a **speed** — the engine ticks each row lasts — where a _SampleToNES_ -project states a tempo and a speed together. The row rate the pair asks for is fractional at -most tempi, so the exporter carries it as a [groove](../glossary.md#groove): whole tick counts, -one per row of a pattern, averaging out to that rate with the longer rows on the bar and the -beat. `sampletones_core/timing/` builds them and in-app playback reads the same groove, so a -document plays the rows the sequencer played. At 60 Hz, speed 6 and tempo 210, a 16-row -pattern in common time comes to +A Bitphase song has a **speed**, the ticks each row lasts. A _SampleToNES_ project has a tempo and a speed +together. The row rate the pair asks for is fractional at most tempi, so the exporter writes it as a +[groove](../glossary.md#groove): whole tick counts, one per row of a pattern, averaging out to that rate, +with the longer rows on the bar and the beat. In-app playback reads the same groove, so a document plays +the rows the sequencer played. For example, at 60 Hz with speed 6 and tempo 210, a 16-row pattern in +common time comes to: ``` 5 4 5 4 5 4 4 4 5 4 4 4 5 4 4 4 69 ticks, a rate of 30/7 per row ``` -**The groove reaches the engine as a table.** A speed effect that names a table reads one of -its entries per pattern row, which is what carries a per-row tick count into a song: +**The groove reaches the engine as a table.** A speed effect that names a table reads one of its entries +per pattern row, and that carries a per-row tick count into a song: | Part | What the exporter writes | | --- | --- | @@ -185,21 +166,19 @@ its entries per pattern row, which is what carries a per-row tick count into a s | The effect | `S` with `delay = 0` and an empty parameter, naming that table | | Its place | the first row of the DPCM channel, in every pattern | -A speed effect applies from whichever channel carries it, so the groove rides the DPCM channel -this exporter leaves silent and every sounding channel keeps the one effect column the chip -gives it. The table advances an entry per row and resumes from where a trigger placed it, so -triggering it again at each pattern start holds every row on the entry that describes it, -however the order jumps. +A speed effect applies from whichever channel has it, so the groove rides the DPCM channel, which this +exporter leaves silent. Every sounding channel keeps the one effect column the chip gives it. The table +advances an entry per row and resumes from where a trigger placed it. Triggering it at each pattern start +therefore holds every row on the entry that describes it, however the order jumps. -**A tempo the speed column states writes neither.** Where every row lasts alike — tempo 150 at -60 Hz, where the rate is the speed itself — `initialSpeed` carries the tempo whole, and the -document holds one table per slice with every effect column empty. +**A tempo the speed column can state needs no groove.** Where every row lasts alike, as with tempo 150 at +60 Hz where the rate is the speed itself, `initialSpeed` has the tempo whole. The document then has one +table per slice, and every effect column is empty. ## E. What the exporter builds per scope -A `.btp` holds a whole document, so every scope lands in one file; a preset holds one -instrument, so a reconstruction lands as a set of them beside the name the export was -given, one per slice. +A `.btp` is a whole document, so every scope lands in one file. A preset is one instrument, so a +reconstruction lands as a set of presets beside the name the export was given, one per slice. | Scope | `.btp` | `.json` preset | | --- | --- | --- | @@ -207,30 +186,27 @@ given, one per slice. | A whole reconstruction | a playable document holding every slice | one file per slice, beside the chosen name | | A project | the song, its samples and its arrangement | — | -**Instrument and reconstruction documents are playable.** Each slice becomes an -instrument and the table that carries its contour, and one pattern triggers every slice -at row 0 on the channel it was reconstructed for, so opening the document and pressing -play sounds the reconstruction. The pattern is sized to cover the longest instrument, -and where one instrument outlasts a single pattern the order gains resting positions -until it has played through. - -**A project flattens its order.** A SampleToNES order frame points each channel at its -own pattern, where a Bitphase order position names one pattern spanning every channel. -Each frame therefore becomes a pattern of its own carrying that frame's channels side -by side, with `patternOrder = [0..n-1]`. The arrangement crosses over whole; it simply -shares fewer patterns. - -Row cells follow from the columns: an instrument command writes the note from -`initial_pitch + transpose`, the instrument number, the table column and the row's -volume; a note-off writes note name `1`; a blank line leaves every column alone. - -**The volume column names silence.** In Bitphase you type `0` to silence a channel and -leave the cell blank to carry its level forward — and the file stores those two as `-1` and -`0`. The volume field is declared `allowZeroValue`, so Bitphase parses a typed `0` to `-1` -and prints a stored `-1` back as `0`, while a stored `0` shows as a blank cell; its engine -reads `-1` as volume zero. So a row asking for silence writes `-1`, a row naming a level -writes it verbatim, and a row with an empty volume cell writes `0` — which is the same cell -you would see in the tracker either way. +**Instrument and reconstruction documents are playable.** Each slice becomes an instrument plus the table +that carries its contour. One pattern triggers every slice at row 0 on the channel it was reconstructed +for, so opening the document and pressing play sounds the reconstruction. The pattern is sized to cover the +longest instrument. Where an instrument outlasts a single pattern, the order gains resting positions until +it has played through. + +**A project flattens its order.** A SampleToNES order frame points each channel at its own pattern, while +a Bitphase order position names one pattern spanning every channel. Each frame therefore becomes a pattern +of its own with that frame's channels side by side, and `patternOrder = [0..n-1]`. The arrangement crosses +over whole and shares fewer patterns. + +Row cells follow from the columns. An instrument command writes the note from `initial_pitch + transpose`, +the instrument number, the table column and the row's volume. A note-off writes note name `1`. A blank +line leaves every column alone. + +**The volume column names silence.** In Bitphase you type `0` to silence a channel and leave the cell +blank to carry its level forward. The file stores those two as `-1` and `0`. The volume field is declared +`allowZeroValue`, so Bitphase parses a typed `0` to `-1` and prints a stored `-1` back as `0`, and a +stored `0` shows as a blank cell. Its engine reads `-1` as volume zero. A row asking for silence therefore +writes `-1`, a row naming a level writes it verbatim, and a row with an empty volume cell writes `0`. Each +is the same cell you would see in the tracker. ## F. Bitphase capacity limits @@ -247,17 +223,16 @@ you would see in the tracker either way. | Speed | 1–255 | the groove's tick counts, bounded to that range | | DPCM channel | present | rests, apart from the groove trigger each pattern's first row carries | -Tables and instruments are numbered together — each slice takes one of each — so the -table column is what a wide document reaches first, and the exporter raises rather than -writing a document whose later voices cannot be named. A song whose rows vary spends one -of those ids on its groove, so the slices a document holds are those the table column can -still name. +Tables and instruments are numbered together, and each slice takes one of each. The table column is +therefore what a wide document reaches first, and the exporter raises an error instead of writing a +document whose later voices cannot be named. A song whose rows vary spends one of those ids on its groove, +so the slices a document holds are those the table column can still name. -## G. What does not cross over +## G. Data without a counterpart -**`ProjectInfo.comment`** has no counterpart in a Bitphase document, which carries a name and -an author only, so the exporter leaves the comment behind. +**`ProjectInfo.comment`** has no counterpart in a Bitphase document, which has a name and an author only, +so the exporter leaves the comment out. -`interruptFrequency` carries the reconstruction's own tick rate. Bitphase's settings -panel offers 50 and 60 Hz, and its loader and timeline accept any value, so a rate -outside that pair plays correctly while leaving that one selector unmatched. +`interruptFrequency` carries the reconstruction's own tick rate. Bitphase's settings panel offers 50 and +60 Hz, and its loader and timeline accept any value. A rate outside that pair plays correctly, and the +panel's selector shows no match. diff --git a/docs/formats/configuration.md b/docs/formats/configuration.md index d86b3029d..bc8faf20e 100644 --- a/docs/formats/configuration.md +++ b/docs/formats/configuration.md @@ -1,19 +1,16 @@ # Configuration file -The generation [configuration](../guide/configuration.md) is stored as a JSON -file, `config.json`, in the documents folder (see -[Where your files live](../guide/files.md)). The interface reads and writes it, -and you can edit it by hand to reach settings the interface does not expose. This -page documents its structure; [Reconstruction algorithms](../concepts/reconstruction.md) -explains what the settings do and lists the defaults. +The generation [configuration](../guide/configuration.md) is stored as a JSON file, `config.json`, in the +documents folder (see [Where your files live](../guide/files.md)). The interface reads and writes it, and +you can edit it by hand to reach settings the interface does not expose. This page documents its +structure. [Reconstruction algorithms](../concepts/reconstruction.md) explains what the settings do. -The file has three sections — `general`, `library`, and `generation` — plus a -`metadata` block that records the application version and is managed -automatically. Unknown keys are rejected, so every key must be one of those below. +The file has three sections, `general`, `library` and `generation`, plus a `metadata` block. The block +records the application version and is managed automatically. Unknown keys are rejected, so every key +must be one of those below. -A key you leave out keeps its shipped value. The shipped values of the -`generation` section are listed in `sampletones_core/configs/generation.yaml` in -the package, which is where they are set for every copy of the program. +A key you leave out keeps its shipped value. The shipped values of the `generation` section are in +`sampletones_core/configs/generation.yaml`, which sets them for every copy of the program. ## `general` @@ -22,7 +19,7 @@ Audio preprocessing and housekeeping. | Key | Meaning | Values | | --- | --- | --- | | `normalize` | normalize the input before matching | `true` / `false` | -| `quantize` | quantize (bit-crush) the input | `true` / `false` | +| `quantize` | quantize the input to fewer volume steps | `true` / `false` | | `quantization_levels` | number of levels when quantizing | integer ≥ 3 | | `min_pitch`, `max_pitch` | lowest and highest pitch the reconstruction may use | 1–127 | | `coefficient_percentile` | percentile of the frames' RMS levels used to set the [working level](../concepts/reconstruction.md) | 0–100 | @@ -47,13 +44,12 @@ change any of these and a different library is selected or generated. ## `generation` -How candidates are scored. It carries one top-level key and groups the scoring -controls into `calculation`, `weights`, `metric`, `decoder`, and `refinement`. +How candidates are scored. It has one top-level key and groups the scoring controls into `calculation`, +`weights`, `metric`, `decoder` and `refinement`. -Which channels a conversion uses, how hard each of them is pushed and how many of -them a recording may sound at once are stated by the conversion itself — per -recording, in the converter — so they are settings of the interface, recorded in -the [stems setup](reconstructions.md) a reconstruction carries. +The channels a conversion uses, how hard each is pushed and how many a recording may sound at once are +set per recording in the converter. The [stems setup](reconstructions.md) a reconstruction carries +records them. | Key | Meaning | Values | | --- | --- | --- | @@ -81,7 +77,7 @@ the [stems setup](reconstructions.md) a reconstruction carries. | `beta` | β for the β-divergence | ≥ 0 | | `perceptual_exponent` | exponent on the loudness weighting | ≥ 0 | | `temporal_level_floor` | floor for the temporal term's normalization, as a share of what one channel plays at full volume | > 0 | -| `silence_floor` | the power a frame quieter than it is measured from, which keeps a silent frame's score finite | > 0 | +| `silence_floor` | the power a frame is measured from when its own bins all lie under it, so a silent frame's score stays finite | > 0 | | `dynamic_range_decibels` | how far under a frame's loudest bin the comparison reaches | > 0 | ### `generation.decoder` @@ -102,9 +98,8 @@ the [stems setup](reconstructions.md) a reconstruction carries. ## Editing the file -Keep to the keys above — unknown keys are rejected. The interface overwrites -`config.json` when you change a setting there, so the keys it does not expose (the -`metric`, `decoder`, phase aligner, and similar) are the ones you will typically -hand-edit. To keep several setups side by side, save them as separate files and -load one with `--config` (see [Command line](../guide/command-line.md)) or from the +Use only the keys above, because unknown keys are rejected. The interface overwrites `config.json` when +you change a setting there. The keys it does not expose, such as `metric`, `decoder` and the phase +aligner, are the ones you typically edit by hand. To keep several setups side by side, save them as +separate files. Load one with `--config` (see [Command line](../guide/command-line.md)) or from the **Reconstruction ▸ Load generation settings...** menu. diff --git a/docs/formats/famitracker.md b/docs/formats/famitracker.md index afc41b613..7ef79c37d 100644 --- a/docs/formats/famitracker.md +++ b/docs/formats/famitracker.md @@ -1,29 +1,26 @@ # FamiTracker export format -This document is the reference for how _SampleToNES_ writes and reads FamiTracker -files. It describes the two binary formats the `sampletones_core.formats.famitracker` -package produces — the `.fti` instrument file and the `.ftm` module file — states what -a voice takes from an `.fti` it reads back, and lists the FamiTracker capacity limits -that the project domain model will grow to respect. - -The target is **vanilla FamiTracker 0.4.6** (`FILE_VER = 0x0440`). Files written to -this specification load in stock FamiTracker as well as the 0CC, Dn-FamiTracker and -FamiStudio forks. The module is single-chip 2A03: five channels (two pulse, triangle, -noise, DPCM), with the DPCM channel and DPCM sample bank always empty by design. - -All multi-byte integers are **little-endian**. Field types below use `uint8`, -`int8`, `uint32`, `int32`; strings are noted per field. Every constant referenced -here has a named counterpart under `sampletones_core/formats/famitracker/specification/` -(grouped by unit: `file`, `blocks`, `channels`, `sequences`, `instruments`, -`patterns`, `parameters`), and every block has its own writer function so this -specification is readable straight from the code. +This document is the reference for how _SampleToNES_ writes and reads FamiTracker files. Read it when you +write or check an `.fti` instrument file or an `.ftm` module file. It covers the binary layout of both +formats (A), the instrument model (B), what an imported `.fti` gives a voice (C), FamiTracker's capacity +limits (D) and the memory an instrument takes in the NSF driver (E). + +The target is **vanilla FamiTracker 0.4.6** (`FILE_VER = 0x0440`). Files written to this specification +load in stock FamiTracker and in the 0CC, Dn-FamiTracker and FamiStudio forks. The module is single-chip +2A03 with five channels: two pulse, triangle, noise and DPCM. The DPCM channel and the DPCM sample bank +are always empty. + +All multi-byte integers are **little-endian**. Field types are `uint8`, `int8`, `uint32` and `int32`. +Strings are noted per field. Every constant named here has a counterpart under +`sampletones_core/formats/famitracker/specification/`, grouped by unit (`file`, `blocks`, `channels`, +`sequences`, `instruments`, `patterns`, `parameters`). Every block has its own writer function, so the +code reads as this specification. ## A. Binary formats ### A.1 `.fti` — instrument file -An `.fti` holds a single 2A03 instrument: its five sequences inline, then an empty -DPCM section. Written by `sampletones_core/formats/famitracker/instrument.py`. +An `.fti` holds a single 2A03 instrument: its five sequences inline, then an empty DPCM section. | Field | Type | Value | | --- | --- | --- | @@ -45,14 +42,12 @@ Each **sequence record**: | item count | `uint32` | number of items | | loop point | `int32` | item index to loop from, or `-1` | | release point | `int32` | item index for note release, or `-1` | -| setting | `uint32` | sequence setting (arpeggio mode etc.); `0` = default | +| setting | `uint32` | sequence setting (arpeggio mode and so on); `0` is the default | | items | `int8` × count | one signed byte per tick | ### A.2 `.ftm` — module file -An `.ftm` is a file header followed by a sequence of named, versioned blocks and a -final `END` marker. Written by `sampletones_core/formats/famitracker/module.py`, one function -per block. +An `.ftm` is a file header, then a sequence of named, versioned blocks, then the `END` marker. **File header** @@ -61,7 +56,7 @@ per block. | magic | 18 bytes | `FamiTracker Module` | | version | `uint32` | `0x0440` | -**Block header** (precedes every block payload) +**Block header** (before every block payload) | Field | Type | Value | | --- | --- | --- | @@ -69,58 +64,119 @@ per block. | version | `int32` | block version | | size | `int32` | payload byte length | -The payload size is known only after the payload is built, so blocks are buffered -before their header is emitted. After the last block, the file ends with the 3-byte -marker `END`. - -**Blocks** (in write order), with their versions: - -- **`PARAMS`** (v6): expansion chip `uint8` (`0` = 2A03/none) · channel count `int32` - (`5`) · machine `int32` (`0` = NTSC, `1` = PAL) · engine speed `int32` (`0` = machine - default, otherwise a refresh rate in Hz) · vibrato style `int32` · highlight first - `int32` · highlight second `int32` · speed split point `int32` (the row where the - tempo/speed interpretation splits, `speed_split_point`). -- **`INFO`** (v1): title, author and copyright, each a fixed **32-byte** NUL-padded - string, in that order. -- **`HEADER`** (v3): track count as `uint8` holding `count − 1`; then each track's - title as a NUL-terminated string; then, for each channel, a channel id `uint8` - followed by one effect-column count per track, each a `uint8` holding `count − 1`. - Channel ids: square1 `0`, - square2 `1`, triangle `2`, noise `3`, DPCM `4`. -- **`INSTRUMENTS`** (v6): instrument count `int32`; then per instrument: index - `int32`, type `uint8` (`1` = 2A03), body, name length `uint32`, name bytes. The 2A03 - body is: sequence count `int32` (`5`); per sequence an enabled `uint8` and a - sequence index `uint8`; then the DPCM key-assignment table across the note range, - all zero here. -- **`SEQUENCES`** (v6): sequence count `int32`. Pass one, per sequence: index `int32`, - type `int32` (volume `0`, arpeggio `1`, pitch `2`, hi-pitch `3`, duty `4`), item - count `uint8`, loop point `int32`, then the items `int8` each. Pass two, per - sequence: release point `int32`, setting `int32`. Instruments reference these - pooled sequences by index — the module stores each sequence once. -- **`FRAMES`** (v3): per song — frame count `int32`, speed `int32`, tempo `int32`, - pattern length `int32`, then the order table: for each frame, one pattern index - `uint8` per channel. -- **`PATTERNS`** (v5): per non-empty pattern — song index `int32`, channel `int32`, - pattern index `int32`, row-item count `int32`; then per stored row: row number - `int32`, note `int8`, octave `int8`, instrument `int8`, volume `int8`, then per - effect column an effect `int8` and a parameter `int8`. -- **`DPCM SAMPLES`** (v1): sample count `uint8` (`0`). -- **`COMMENTS`** (v1): display-on-open flag `int32`, then the comment as a - NUL-terminated string. - -**Pattern cell encoding.** Note: `0` = empty, `1`–`12` = C–B, `13` = release, -`14` = halt (note cut); octave `0`–`7`. Empty instrument `0x40`, empty volume `0x10`, -empty effect `0`. A pitch converts to a cell by `note = pitch % 12 + 1` and -`octave = pitch // 12 − 2`, matching `pitch_to_name` in -`sampletones_core/utils/frequencies.py`. +The payload size is known once the payload is built, so each block is buffered before its header is +written. After the last block comes the 3-byte marker `END`. + +**Blocks**, in write order: + +| Block | Version | Payload | +| --- | --- | --- | +| `PARAMS` | 6 | The parameters table below | +| `INFO` | 1 | Title, author and copyright, each a 32-byte NUL-padded string, in that order | +| `HEADER` | 3 | The header table below | +| `INSTRUMENTS` | 6 | The instruments table below | +| `SEQUENCES` | 6 | The sequences table below | +| `FRAMES` | 3 | The frames table below | +| `PATTERNS` | 5 | The patterns table below | +| `DPCM SAMPLES` | 1 | Sample count `uint8`, always `0` | +| `COMMENTS` | 1 | Display-on-open flag `int32`, then the comment as a NUL-terminated string | + +`PARAMS` payload: + +| Field | Type | Value | +| --- | --- | --- | +| expansion chip | `uint8` | `0` (2A03, no expansion) | +| channel count | `int32` | `5` | +| machine | `int32` | `0` NTSC, `1` PAL | +| engine speed | `int32` | `0` for the machine's default, otherwise a refresh rate in Hz | +| vibrato style | `int32` | | +| highlight first | `int32` | | +| highlight second | `int32` | | +| speed split point | `int32` | the row where the tempo and speed interpretation splits (`speed_split_point`) | + +`HEADER` payload: + +| Field | Type | Notes | +| --- | --- | --- | +| track count | `uint8` | the count minus 1 | +| track title, one per track | NUL-terminated string | | +| channel id, one per channel | `uint8` | square1 `0`, square2 `1`, triangle `2`, noise `3`, DPCM `4` | +| effect-column count, one per track after each channel id | `uint8` | the count minus 1 | + +`INSTRUMENTS` payload: + +| Field | Type | Notes | +| --- | --- | --- | +| instrument count | `int32` | | +| index, per instrument | `int32` | | +| type | `uint8` | `1` (2A03) | +| body | — | the 2A03 body below | +| name length | `uint32` | | +| name | bytes | | + +The 2A03 body is the sequence count `int32` (`5`), then per sequence an enabled `uint8` and a sequence +index `uint8`. The DPCM key-assignment table across the note range follows, all zero. + +`SEQUENCES` payload: + +| Pass | Field | Type | Notes | +| --- | --- | --- | --- | +| | sequence count | `int32` | | +| 1, per sequence | index | `int32` | | +| 1 | type | `int32` | volume `0`, arpeggio `1`, pitch `2`, hi-pitch `3`, duty `4` | +| 1 | item count | `uint8` | | +| 1 | loop point | `int32` | | +| 1 | items | `int8` each | | +| 2, per sequence | release point | `int32` | | +| 2 | setting | `int32` | | + +Instruments reference the pooled sequences by index, so the module stores each sequence once. + +`FRAMES` payload, per song: + +| Field | Type | Notes | +| --- | --- | --- | +| frame count | `int32` | | +| speed | `int32` | | +| tempo | `int32` | | +| pattern length | `int32` | | +| order table | `uint8` | for each frame, one pattern index per channel | + +`PATTERNS` payload, per non-empty pattern: + +| Field | Type | Notes | +| --- | --- | --- | +| song index | `int32` | | +| channel | `int32` | | +| pattern index | `int32` | | +| row-item count | `int32` | | +| row number | `int32` | starts each stored row | +| note | `int8` | per stored row | +| octave | `int8` | per stored row | +| instrument | `int8` | per stored row | +| volume | `int8` | per stored row | +| effect | `int8` | per stored row, one for each effect column | +| effect parameter | `int8` | per stored row, one for each effect column | + +**Pattern cell encoding** + +| Field | Values | +| --- | --- | +| note | `0` empty, `1`–`12` C–B, `13` release, `14` halt (note cut) | +| octave | `0`–`7` | +| instrument | `0x40` when empty | +| volume | `0x10` when empty | +| effect | `0` when empty | + +A pitch converts to a cell by `note = pitch % 12 + 1` and `octave = pitch // 12 − 2`. This matches +`pitch_to_name` in `sampletones_core/utils/frequencies.py`. ## B. The 2A03 instrument -Both file formats describe the same instrument model. A 2A03 instrument is a name -plus five **sequences**, one per dimension, advanced one item per engine tick while a -note sounds. The `.fti` file stores the sequences inline; the `.ftm` module pools -them in the `SEQUENCES` block and references them by index, so identical sequences -are stored once. +Both file formats describe the same instrument model. A 2A03 instrument is a name plus five +**sequences**, one per dimension. Each sequence advances one item per engine tick while a note sounds. +The `.fti` file stores the sequences inline. The `.ftm` module pools them in the `SEQUENCES` block and +references them by index, so identical sequences are stored once. The five sequence kinds, in slot order (`SequenceKind` in `specification/sequences.py`): @@ -132,142 +188,126 @@ The five sequence kinds, in slot order (`SequenceKind` in `specification/sequenc | 3 | Hi-pitch | per-tick divider offset, sixteen steps per unit | | 4 | Duty / Noise | pulse duty cycle 0–3, or the noise short/long mode | -Each sequence carries: - -- **items** — the signed per-tick values (`int8`); -- **loop point** — the item index playback returns to after the last item, or `-1` - to stop at the end; -- **release point** — the item index playback jumps to when the note is released, - or `-1` for none; -- **setting** — the sequence mode; for arpeggio, `0` selects absolute (the offsets - are added to the played note). - -**The bend and the arpeggio that pins it.** FamiTracker walks an instrument's sequences in -slot order, and an arpeggio in absolute mode reloads the period from the note before the two bend -sequences add to it (`CSeqInstHandler::ProcessSequence`). While the arpeggio runs, a pitch item is -therefore an *offset from the note* for that tick, and a hi-pitch item is the same offset counted -sixteen dividers at a time; once the arpeggio halts, the same items start accumulating on the -running period instead. _SampleToNES_ writes and reads a bend as the per-tick offset, so the writer -guarantees the arpeggio that makes that reading hold: an instrument writing a bend and no arpeggio -gains one holding a single zero at loop point 0, and a shorter arpeggio is brought to the bend's -length holding its final note. What one step is worth follows the note it bends — under a cent at -the lowest notes, widening to a whole semitone at the highest, where the divider grid is already -coarser than the note grid. - -**Looping.** Each sequence states the item it repeats from, so a held note sustains from -that item on. A dimension written without one leaves its loop point at `-1` and plays its -items once. Every envelope carries its own point, so a two-item duty cycle circles on its -own period beside a longer volume envelope. A point beyond a sequence's own items repeats -its final item, which is the value it would hold anyway. - -**Lengths.** FamiTracker advances each sequence on its own per-tick counter. A sequence -that reaches its last item halts and leaves the value it wrote applied, which the driver -holds for as long as the note sounds (`CSeqInstHandler::UpdateInstrument`). Every -dimension therefore carries the length it was written at: a two-item volume envelope -beside a one-item duty envelope plays exactly as a padded pair would, and costs the -padding less. - -**The release.** A volume envelope whose frames end audible carries one silent item past -them, and that item is what stops the note: the driver holds a halted sequence's last value -for as long as a row keeps the note sounding, so a volume envelope ending audible would sound -to the end of the song. Every generator writes that item, so a volume dimension runs one item -longer than the frames it describes. - -**The item limit.** A FamiTracker sequence holds 252 items, and that ceiling belongs to this -writer: an envelope carries whatever length it was written at, and meets the limit only here. -A dimension over it is written as its opening items, and a volume dimension keeps its release -as the last of them — the note has to end, so the release displaces the sounding item that -would not fit. A reconstruction therefore reaches the limit at 252 frames, since its volume -carries the release past them; that is 8.4 s at the default 30 fps. The export reports what it -left out, and the instruments panel colors a sequence input warning orange while a file would -hold only part of it, so the limit is visible before an export. - -An empty dimension is written as a disabled sequence, which is a different instrument from -one carrying a single zero: the disabled slot leaves that dimension to the channel, while a -one-item sequence sets the value once and holds it. A dimension arrives empty when the -reconstruction records it as one the channel governs — the state clearing the envelope in the -instruments panel puts it in (see [Reconstructions](reconstructions.md)). - -**How _SampleToNES_ fills an instrument.** Each channel slice of a sample's -reconstruction becomes one instrument, so a sample yields one to four instruments. -A reconstruction holds a stream for every channel, and one describing no frame is a -channel standing by (see [Reconstructions](reconstructions.md#contents)): it takes no -place in the instrument table, so the instruments an export writes are the channels -that play. -The arpeggio sequence carries the reconstruction's pitch contour as signed offsets, -and triggering the instrument at `initial_pitch` replays that contour. Volume, duty -(or noise mode) and the two bend sequences carry across directly; a conversion that bent no note -records both bend dimensions as ones the channel governs, so they reach the file as disabled slots. -The DPCM key-assignment table is empty by design. - -An [instrument](../glossary.md#instrument) written by hand is one set of envelopes -every channel reads, which is the instrument model FamiTracker itself uses, so it -becomes a single instrument however many channels play it. Each dimension is written at -the length it was typed at, and each states its own loop point, so a tracker advancing -every sequence on a counter of its own sounds it the way the engine here plays it. Every -channel that names it reaches that one instrument, each against the initial pitch it -reads — its note on the tonal channels, its period on noise. - -**Where a row's note comes from.** A voice states where its zero is and a row states -the step from it, so a pattern cell holds `reference + transpose`, held inside the -range a tonal channel plays and wrapped into the sixteen periods on noise. A sample's -reference is the offset origin its conversion chose; a hand-written instrument's is the -initial pitch it states. - -That origin is chosen once, when the reconstruction is built, and stored with it as -that channel's reference pitch (see [Reconstructions](reconstructions.md#contents)). -For the pitched channels `center_pitch` picks it, taking the midpoint of the contour's -`(lowest, highest)` range; the noise channel takes the first sounding period. Every -later export reports that stored pitch as `initial_pitch` and writes each frame as -`pitch − initial_pitch`, wrapped into the 16 available periods on noise. The offsets -straddle zero and stay compact around one note, and the pattern cell holds the -contour's midpoint — a rising contour prints its middle note and opens below it. +Each sequence has: + +| Part | Meaning | +| --- | --- | +| items | the signed per-tick values (`int8`) | +| loop point | the item index playback returns to after the last item, or `-1` to stop at the end | +| release point | the item index playback jumps to when the note is released, or `-1` for none | +| setting | the sequence mode; for arpeggio, `0` selects absolute (the offsets are added to the played note) | + +**Bend and arpeggio.** FamiTracker walks an instrument's sequences in slot order. An arpeggio in absolute +mode reloads the period from the note before the two bend sequences add to it +(`CSeqInstHandler::ProcessSequence`). While the arpeggio runs, a pitch item is an offset from the note for +that tick, and a hi-pitch item is the same offset counted sixteen dividers at a time. Once the arpeggio +halts, the same items accumulate on the running period. + +_SampleToNES_ writes and reads a bend as the per-tick offset. The writer therefore keeps an arpeggio +running for as long as the bend. An instrument with a bend and no arpeggio gets one holding a single zero +at loop point 0. A shorter arpeggio is extended to the bend's length by holding its final note. What one +step is worth follows the note it bends: under a cent at the lowest notes, widening to a whole semitone at +the highest, where the divider grid is already coarser than the note grid. + +**Looping.** Each sequence has a loop point: the item it repeats from while a note is held. A sequence +written without one has the loop point `-1` and plays its items once. Every sequence has its own point, so +a two-item duty cycle can circle on its own period beside a longer volume envelope. A point beyond a +sequence's own items repeats its final item, which is the value it would hold anyway. + +**Lengths.** FamiTracker advances each sequence on its own per-tick counter. A sequence that reaches its +last item halts, and the value it wrote stays applied. The driver holds that value for as long as the note +sounds (`CSeqInstHandler::UpdateInstrument`). Every dimension therefore keeps the length it was written +at. A two-item volume envelope beside a one-item duty envelope plays exactly as a padded pair would, and +costs less than the padding. + +**The release.** A volume envelope whose frames end audible gets one silent item after them, and that item +stops the note. The driver holds a halted sequence's last value for as long as a row keeps the note +sounding, so a volume envelope that ended audible would sound to the end of the song. Every generator +writes the release item, so a volume dimension is one item longer than the frames it describes. + +**The item limit.** A FamiTracker sequence holds up to 252 items. Only this writer applies the limit: an +envelope keeps whatever length it was written at until export. A dimension over the limit is written as +its opening items. A volume dimension keeps its release as the last item, because the note has to end, so +the release displaces the last sounding item that would not fit. A reconstruction reaches the limit at 252 +frames, since its volume carries the release past them. At the default 30 fps that is 8.4 s. The export +reports what it left out. + +**Empty dimensions.** An empty dimension is written as a disabled sequence. This differs from a sequence +with a single zero: a disabled slot leaves that dimension to the channel, while a one-item sequence sets +the value once and holds it. A dimension is empty when the reconstruction records it as one the channel +governs. Clearing the envelope in the instruments panel produces that state (see +[Reconstructions](reconstructions.md)). + +**How _SampleToNES_ fills an instrument.** Each channel slice of a sample's reconstruction becomes one +instrument, so a sample yields one to four instruments. A reconstruction has a stream for every channel. +A stream that describes no frame is a channel standing by (see +[Reconstructions](reconstructions.md#contents)). It gets no place in the instrument table, so the +instruments an export writes are the channels that play. + +The arpeggio sequence carries the reconstruction's pitch contour as signed offsets, and triggering the +instrument at `initial_pitch` replays that contour. Volume, duty (or noise mode) and the two bend +sequences carry across directly. A conversion that bent no note records both bend dimensions as ones the +channel governs, so they reach the file as disabled slots. The DPCM key-assignment table is always empty. + +An [instrument](../glossary.md#instrument) written by hand is one set of envelopes that every channel +reads, as in FamiTracker itself. It becomes a single instrument, however many channels play it. Each +dimension is written at the length it was typed at and has its own loop point, so a tracker that advances +every sequence on its own counter sounds it the way the engine here plays it. Every channel that names the +instrument reaches that one instrument, each against the initial pitch it reads: its note on the tonal +channels, its period on noise. + +**Where a row's note comes from.** A voice has a reference, the place where its zero is, and a row has a +step from it. A pattern cell therefore holds `reference + transpose`, kept inside the range a tonal +channel plays and wrapped into the sixteen periods on noise. A sample's reference is the offset origin its +conversion chose. A hand-written instrument's reference is its initial pitch. + +The conversion chooses the origin once, when the reconstruction is built, and stores it as that channel's +reference pitch (see [Reconstructions](reconstructions.md#contents)). For the pitched channels +`center_pitch` picks it, as the midpoint of the contour's `(lowest, highest)` range. The noise channel +takes the first sounding period. Every later export reports the stored pitch as `initial_pitch` and writes +each frame as `pitch − initial_pitch`, wrapped into the 16 available periods on noise. The offsets +straddle zero and stay compact around one note. The pattern cell holds the contour's midpoint, so a rising +contour prints its middle note and opens below it. ## C. Reading an instrument file -An `.fti` is read as well as written: **Import instrument...** in the sequencer brings -one into the voice pool as a hand-written [instrument](../glossary.md#instrument). -`instrument.py::read_fti` parses the layout in section A.1, and -`voice.py::instrument_to_voice` makes a voice of the 2A03 instrument it holds. +An `.fti` is read as well as written. **Import instrument...** in the sequencer brings one into the voice +pool as a hand-written [instrument](../glossary.md#instrument). `instrument.py::read_fti` parses the +layout in section A.1, and `voice.py::instrument_to_voice` makes a voice from the 2A03 instrument it +holds. -A voice carries all five dimensions — volume, arpeggio, pitch, hi-pitch and duty — each with -the item it repeats from, so those come across as they stand. The voice takes the name the -file states, and a file naming nothing leaves the voice named after the file itself. The -arpeggio is read as offsets from the pitch a hand-written voice rests at, since a tracker +A voice has all five dimensions, each with the item it repeats from, so they come across as they stand. +The voice takes the name in the file. A file without a name leaves the voice named after the file itself. +The arpeggio is read as offsets from the pitch a hand-written voice rests at, because a tracker instrument sounds at whatever note a row names it with. -A sequence looping from one of its items gives that dimension the point; one halting at -its end leaves the dimension playing its items once, holding the last of them for as long -as the note sounds. A point outside the items the sequence carries is read as no point. +A sequence that loops from one of its items gives that dimension the point. A sequence that halts at its +end leaves the dimension playing its items once, holding the last of them for as long as the note sounds. +A point outside the sequence's items is read as no point. -**What the voice leaves to the file.** A tracker instrument states more than a voice -holds, and each of those is reported once the import lands, so a reader learns what the -file carried (`InstrumentOmission` in `voice.py`): +**Settings the voice does not hold.** A tracker instrument can say more than a voice holds. The import +reports each of these once it lands, so a reader learns what the file carried: -| Stated in the file | What the voice holds | +| In the file | In the voice | | --- | --- | -| a bend outrunning its arpeggio | each item as the offset it states, where the tracker would accumulate it past the arpeggio's last tick (section B) | +| a bend outrunning its arpeggio | each item as the offset it says, where the tracker would accumulate it past the arpeggio's last tick (section B) | | a release point | a note the pattern cuts with a note-off | | an arpeggio in fixed, relative or scheme mode | absolute offsets | -Each of these is a dimension the project model will grow to hold; `bugs-and-todos.md` -under **Tracker** owns that list. +`bugs-and-todos.md` lists these under **Tracker**. -A sequence carrying an item outside the range its dimension holds raises -`InvalidInstrumentValuesError`. The file is read before the pool is touched, so a file -the reader cannot take leaves the project as it stood and the history without an entry. +A sequence with an item outside the range its dimension holds raises `InvalidInstrumentValuesError`. The +file is read before the pool is touched, so a rejected file leaves the project and its history unchanged. ## D. FamiTracker capacity limits -FamiTracker bounds several quantities, and those bounds belong to this writer. A project -holds what a reader wrote — an envelope of any length, a pool of any size — and meets a -limit where a file is built, so the editor stays free of a format it may never export to. -The writer guards each limit, so every file it writes loads: it raises on a project -structure FamiTracker has no room for, and shortens an envelope that outruns a sequence -while keeping the release that ends its note. What each limit costs is reported to the -reader; this table is where those answers are stated. +FamiTracker bounds several quantities, and only this writer applies those bounds. A project keeps whatever +a reader wrote, such as an envelope of any length or a pool of any size, and meets a limit when a file is +built. The writer guards each limit, so every file it writes loads. It raises on a project structure +FamiTracker has no room for, and it shortens an envelope that outruns a sequence while keeping the release +that ends its note. -| Quantity | FamiTracker limit | Project bound today | Exporter behavior | +| Quantity | FamiTracker limit | Project bound | Exporter behavior | | --- | --- | --- | --- | | Instruments | 64 total | unbounded (1–4 per sample, one per hand-written instrument) | raises when the instruments exceed 64 | | Sequences per kind | 128 | unbounded | raises when a kind's pool exceeds 128 | @@ -279,25 +319,21 @@ reader; this table is where those answers are stated. | Title / author | 32 bytes each | 64 characters | truncates to 32 bytes | | Comment | free text (COMMENTS block) | 65536 characters | carried in full | | Tempo / speed | engine-dependent (split at row `speed_split_point`) | tempo 32–255, speed 1–31 | written verbatim from settings | -| DPCM samples | 64 | not modeled | always empty by design | +| DPCM samples | 64 | not modeled | always empty | -The exporter also reserves a per-channel empty pattern index (`max used index + 1`) -for order slots the song leaves unset; a channel that already fills indices up to -127 leaves no room for it, which the exporter reports rather than emitting a corrupt -order. When the domain model grows to enforce these limits, the editor can prevent -reaching a state the exporter would reject. +The exporter also reserves an empty pattern index per channel (`max used index + 1`) for order slots the +song leaves unset. A channel that already fills indices up to 127 leaves no room for it, and the exporter +reports this instead of writing a corrupt order. ## E. Driver memory footprint -Compiling a module into an NSF lays each instrument out across two regions of the driver's -data, and an instrument's sequences size both of them. `footprint.py` measures the two, and -`specification/memory.py` names every field the measurement counts. The instruments panel and -the samples context menu display the result, so the cost of a sample is readable before an -export. +Compiling a module into an NSF lays each instrument out across two regions of the driver's data. An +instrument's sequences size both regions. The application shows the size before an export, so the cost of +a sample is visible in advance. -The **instrument region** holds the instrument list — one pointer per instrument — followed by -each instrument's body: a sequence-enable bitmask, then one pointer per populated sequence. The -**sequence region** holds one chunk per sequence: a four-field header followed by the items. +The **instrument region** holds the instrument list, one pointer per instrument, followed by each +instrument's body: a sequence-enable bitmask, then one pointer per populated sequence. The **sequence +region** holds one chunk per sequence: a four-field header followed by the items. | Field | Bytes | Region | | --- | --- | --- | @@ -307,29 +343,26 @@ each instrument's body: a sequence-enable bitmask, then one pointer per populate | item count · loop point · release point · setting | 1 each | sequence | | item, per tick | 1 | sequence | -An instrument with `n` populated sequences carrying `s₁ … sₙ` items therefore occupies -`3 + 2n` bytes of the instrument region and `Σ (4 + sᵢ)` of the sequence region. A dimension the -channel leaves unused is written as a disabled slot, and the populated sequences alone are -charged: a reconstruction that bent no note charges 3 sequences on the pulse and noise channels -(volume, arpeggio, duty) and 2 on triangle, and each bend an instrument writes adds one more. -Each sequence is charged at its own length (section B), so shortening any one dimension shows -in the figure, and an instrument tops out at 777 bytes — three sequences at the 252-item limit. - -These two figures are the ones FamiTracker itself prints while creating an NSF — -`Instruments used: N (X bytes)` and `Sequences used: M (Y bytes)` — which is how a measurement -is held against the tracker. - -**Version.** The figures are vanilla FamiTracker 0.4.6, the target section A names. The 0CC and -Dn-FamiTracker forks open each instrument body with a channel-type byte, so an instrument costs -one byte more there. - -**Pooling narrows a module's total.** The `SEQUENCES` block stores each distinct sequence once -(section A.2), so a module holding two instruments with the same volume envelope pays for that -chunk once. A per-instrument or per-sample figure states that instrument's own cost, and a -module total is therefore at most the sum of them. Within one instrument each kind appears -once, so its own sequences are charged once each. - -**Every dimension is charged at its own length.** A sequence is written at the length it holds -(section B), so a figure counts each dimension as it stands and the loop point one of them -carries adds a byte, not a padding. What a voice is measured at is therefore what its -**Export instrument...** writes. +An instrument with `n` populated sequences carrying `s₁ … sₙ` items therefore occupies `3 + 2n` bytes of +the instrument region and `Σ (4 + sᵢ)` of the sequence region. A dimension the channel leaves unused is +written as a disabled slot, and only populated sequences are charged. A reconstruction that bent no note +charges 3 sequences on the pulse and noise channels (volume, arpeggio, duty) and 2 on triangle. Each bend +an instrument writes adds one more. Each sequence is charged at its own length (section B), so shortening +any one dimension shows in the figure. An instrument tops out at 777 bytes: three sequences at the +252-item limit. + +FamiTracker itself prints these two figures while creating an NSF, as `Instruments used: N (X bytes)` and +`Sequences used: M (Y bytes)`. A measurement can be checked against them. + +**Version.** The figures are for vanilla FamiTracker 0.4.6, the target named at the top of this document. +The 0CC and Dn-FamiTracker forks open each instrument body with a channel-type byte, so an instrument +costs one byte more there. + +**Pooling.** The `SEQUENCES` block stores each distinct sequence once (section A.2), so two instruments +with the same volume envelope pay for that chunk once. A per-instrument or per-sample figure is that +instrument's own cost, so a module total is at most the sum of them. Within one instrument each kind +appears once, so its own sequences are charged once each. + +**Length.** A sequence is written at the length it holds (section B), so a figure counts each dimension as +it stands. A loop point on one dimension adds a byte and no padding. The figure for a voice is therefore +what its **Export instrument...** writes. diff --git a/docs/formats/instruction-libraries.md b/docs/formats/instruction-libraries.md index fedcc4343..e9be49e9a 100644 --- a/docs/formats/instruction-libraries.md +++ b/docs/formats/instruction-libraries.md @@ -1,19 +1,16 @@ # Instruction libraries -An instruction library is stored as a single `.ins` file holding, for every -possible instruction, the waveform its channel produces and that waveform's -[spectrum](../glossary.md#spectrum-feature-histogram). It is the catalog the -reconstruction search draws its candidates from. For what a library is and how -it is built, see [Instruction library](../concepts/instruction-library.md); this -page documents the file. +An instruction library is stored as a single `.ins` file. It has, for every possible instruction, the +waveform its channel produces and that waveform's [spectrum](../glossary.md#spectrum-feature-histogram). +It is the catalog the reconstruction search draws its candidates from. For what a library is and how it is +built, see [Instruction library](../concepts/instruction-library.md). This page documents the file. Libraries are generated from the _Instructions_ tab (or with `sampletones library`) and stored in the documents folder. ## Contents -A library is keyed by the configuration that produces it and holds one entry per -instruction. +A library is keyed by the configuration that produces it and has one entry per instruction. ### Per-instruction data @@ -32,17 +29,15 @@ Each entry contains: ### Configuration key -Each library corresponds to one configuration. The parameters that change the -rendered waveforms or their spectra — sample rate, NES frequency, FFT window -size, transformation gamma, and spectrum method — form its key, so changing any -of them selects (or generates) a different library. What gamma and the spectrum -method mean is covered in [Reconstruction algorithms](../concepts/reconstruction.md) -(§3.2–3.3). +Each library corresponds to one configuration. The parameters that change the rendered waveforms or their +spectra form its key: sample rate, NES frequency, FFT window size, transformation gamma and spectrum +method. Changing any of them selects, or generates, a different library. +[Reconstruction algorithms](../concepts/reconstruction.md) explains gamma and the spectrum method. ## File format -Libraries are stored as `.ins` files in the documents folder, with the -configuration embedded in the file name: +A file holds a deflated [MessagePack](https://msgpack.org/) payload, with the framing described in +[Reconstructions](reconstructions.md#storage-and-export), and has its configuration in the file name: ``` sr_44100_nf_60_ws_13579_tg_0_sm_cqt_ch_384e710987cb958adf2b214df1267d10.ins @@ -59,18 +54,15 @@ sr_44100_nf_60_ws_13579_tg_0_sm_cqt_ch_384e710987cb958adf2b214df1267d10.ins ## Versioning -Each file records the library data-version it was written with, in the metadata -that leads the file, so the version reads from the first bytes without loading -the entries. A library is derived data: its settings and the generators -determine it wholly. A library written at the version this build writes is used -as it stands, and any other is rebuilt from its settings the first time it is -needed — a conversion rebuilds it unprompted, and opening one from the -_Instructions_ tab asks first (see -[Data compatibility](../development/release/compatibility.md)). - -The version therefore names what generation produces: a change to the -generators or to feature extraction bumps it, which is what has every stored -library rebuilt. Installs of different versions that share one library folder -rebuild each other's libraries as each needs them. +Each file records the library data version it was written with, in the metadata that leads the file, so +the version reads from the first bytes without loading the entries. A library is derived data: its +settings and the generators determine it wholly. A library written at the version this build writes is +used as it stands. Any other is rebuilt from its settings the first time it is needed. A conversion +rebuilds it without asking, and opening one from the _Instructions_ tab asks first. See +[Data compatibility](../development/release/compatibility.md). + +The version names what generation produces. A change to the generators or to feature extraction bumps +it, and every stored library is then rebuilt. Installs of different versions that share one library +folder rebuild each other's libraries as each needs them. The current data version is 2.1. diff --git a/docs/formats/nsf.md b/docs/formats/nsf.md index 61fb9a5a0..b85cab9ed 100644 --- a/docs/formats/nsf.md +++ b/docs/formats/nsf.md @@ -1,23 +1,19 @@ # NSF export format -This document is the reference for the `.nsf` files _SampleToNES_ writes: the file a -console or an NSF player loads, and the song block inside it that the player's own 6502 -driver reads. Read it before changing anything under `sampletones_player/nsf/`, -`sampletones_player/compression/`, or the assembly under `sampletones_player/driver/`. -The design behind the format — why a song is stored this way and how the driver is held -to it — is in [the player](../development/player.md), and the compression scheme is explained -in [song compression](../concepts/compression.md); the layout itself is here. - -An `.nsf` is unlike the tracker exports beside it. A [FamiTracker](famitracker.md) or -[Bitphase](bitphase.md) file describes a song to a program that already knows how to play -one; an `.nsf` carries its own player. The file therefore holds three things: a header -naming where the program loads and which routines the console calls, the assembled driver, -and the song that driver plays. - -Every constant named here has a counterpart under -`sampletones_player/specification/`, and the assembly reads the same figures from -`sampletones_tools/player/assembly/include/song.inc`. The two are held against each other by a test, so a -change made in one file and forgotten in the other is reported by name. +This document is the reference for the `.nsf` files _SampleToNES_ writes: the file a console or an NSF +player loads, and the song block inside it that the player's own 6502 driver reads. Read it before +changing how an `.nsf` is written or read. [The player](../development/player.md) explains why a song is +stored this way and how the driver is held to it. [Song compression](../concepts/compression.md) explains +the compression scheme. The layout is here. + +An `.nsf` differs from the tracker exports beside it. A [FamiTracker](famitracker.md) or +[Bitphase](bitphase.md) file describes a song to a program that already knows how to play one, and an +`.nsf` carries its own player. The file has three parts: a header, the assembled driver, and the song the +driver plays. The header names where the program loads and which routines the console calls. + +Every constant named here has a counterpart under `sampletones_player/specification/`. The assembly reads +the same figures from `sampletones_tools/player/assembly/include/song.inc`. A test holds the two against +each other, so a change made in one file and forgotten in the other is reported by name. ## A. The file @@ -27,39 +23,35 @@ change made in one file and forgotten in the other is reported by name. +128 + driver length the song block, at the address the header states ``` -The header is NSF version 1: the magic `NESM\x1a`, one song, the load, init and play -addresses, three 32-byte text fields, and the NTSC play period. The text fields carry the -title, the artist and the copyright an export states. Each is UTF-8 ending in a NUL, so it -holds `STRING_TEXT_SIZE` bytes of text, cut on a character boundary; `nsf/information.py` -states that cut, and `nsf/header.py` writes the header. +The header is NSF version 1. It has the magic `NESM\x1a`, one song, the load, init and play addresses, +three 32-byte text fields, and the NTSC play period. The text fields have the title, the artist and the +copyright an export sets. Each is UTF-8 ending in a NUL, so it holds `STRING_TEXT_SIZE` bytes of text, cut +on a character boundary. `nsf/information.py` does the cut, and `nsf/header.py` writes the header. -The driver's entry points lead its image as a pair of jumps, so `init` answers at the load -address and `play` three bytes later whatever the driver's own length. That is what lets -the header state both addresses without assembling anything. The song follows the code -directly, which is the one address a build decides — `driver/addresses.py` reads it back -out of the linker's own labels. +The driver's image starts with two jumps, the entry points. `init` is at the load address and `play` is +three bytes later, whatever the driver's own length. The header can therefore give both addresses without +assembling anything. The song follows the code directly. That address is the one thing a build decides, +and `driver/addresses.py` reads it back from the linker's own labels. -The console calls `init` once and then `play` once a video frame. The header asks for the -NTSC frame period, so a player honoring the field and one driving from the frame itself -run a song at the speed it was built at. +The console calls `init` once and then `play` once per video frame. The header asks for the NTSC frame +period. A player that honors the field and one that drives from the frame itself both run a song at the +speed it was built at. -**The program area is 32 KB**, from `$8000` upward, and the song block has whatever the -driver leaves of it. A song that outgrows that space is reported as an export failure -rather than written short. +**The program area is 32 KB**, from `$8000` upward, and the song block gets what the driver leaves of it. +A song that outgrows that space is reported as an export failure, and no shortened file is written. ## B. The song block -A song reaches the console as one **token stream** per plane, decoded a tick at a time -against a **dictionary** of phrases and a **timer table** of pitches. Every channel writes -a control plane and a value plane, and a tone channel writes a bend plane besides. Every -offset below is a `uint16` counted from the block's own first byte, so the whole block -plays from wherever the file loads it. +A song reaches the console as one **token stream** per plane. The driver decodes each stream a tick at a +time against a **dictionary** of phrases and a **timer table** of pitches. Every channel has a control +plane and a value plane, and a tone channel has a bend plane besides. Every offset below is a `uint16` +counted from the block's first byte, so the whole block plays from wherever the file loads it. ``` +0 header -+55 timer table ++51 timer table phrase table: count, then one offset per phrase - phrase bodies: each a length byte, then its values + phrase bodies: each a length byte, a count byte, then its values one token stream per plane, in plane order ``` @@ -74,81 +66,88 @@ plays from wherever the file loads it. | +7 | 2 | where the timer table begins | | +9 | 2 | where the phrase table begins | | +11 | `PLANE_COUNT`×2 | where each plane's stream begins, or `$FFFF` for an absent plane | -| +33 | `PLANE_COUNT`×2 | where each plane's stream is re-entered once the song comes round, or `$FFFF` for an absent plane | +| +31 | `PLANE_COUNT`×2 | where each plane's stream is re-entered once the song comes round, or `$FFFF` for an absent plane | All fields are little-endian, and the header runs to `SONG_HEADER_SIZE` bytes. -**The step is how one data set plays at every rate.** A reconstruction advances its -envelopes at whatever rate it was built at, and the console calls `play` at the video -frame rate. The step is the first measured against the second, held as a whole byte and a -16-bit fraction; the driver adds it to an accumulator each call and advances the streams -by the whole ticks that fall out. A song slower than the play rate stands still on the -calls between its ticks, and a faster one advances several. +**The step lets one data set play at every rate.** A reconstruction advances its envelopes at the rate it +was built at, and the console calls `play` at the video frame rate. The step is the first rate measured +against the second, held as a whole byte and a 16-bit fraction. The driver adds it to an accumulator on +each call and advances the streams by the whole ticks that fall out. A song slower than the play rate +stands still on the calls between its ticks, and a faster one advances several. ### B.2 The timer table -The table holds the timer register value every pitch sounds at: the low byte of each -pitch, in pitch order, then the high byte of each. One pointer reaches both halves, which -is what the driver's lookup takes advantage of. +The table has the timer register value each pitch sounds at: the low byte of each pitch in pitch order, +then the high byte of each. One pointer reaches both halves, which the driver's lookup uses. -A plane names a pitch as its **index** — the distance above the lowest pitch the tuning -covers — rather than as a divider. A tick's divider is written as the index of the pitch it -is counted from, beside a bend of the steps from that pitch's own divider (§C). Pitches -beyond the divider's range share the timer they clamp to, and the lowest pitch sounding a -timer stands for the whole group. +A plane names a pitch as its **index**, the distance above the lowest pitch the tuning covers, and not as +a divider. A tick's divider is written as the index of the pitch it is counted from, beside a bend: the +steps from that pitch's own divider (section C). Pitches beyond the divider's range share the timer they +clamp to, and the lowest pitch sounding a timer represents the whole group. -The table is written from the tuning the exported work was built at, computed by the very -function the reconstruction's own generators render from. +The table comes from the tuning the exported work was built at, computed by the function the +reconstruction's own generators render from. ### B.3 The dictionary ``` count 1 byte, how many phrases the table holds offsets one uint16 per phrase, in id order -bodies each phrase: a length byte, then its values +bodies each phrase: a length byte, a count byte, then its values ``` -A **phrase** is a run of values a plane plays, stored at the pitch it was found at. Its -position in the table is its **id**, and the ids that ride inside a token's opcode are the -cheap ones, so the phrases a song leans on hardest are listed first. +A **phrase** is a run of values a plane plays, stored at the pitch it was found at. Its position in the +table is its **id**. The ids that fit inside a token's opcode are the cheap ones, so the phrases a song +relies on most are listed first. + +**A phrase states the count its tokens play it at most often.** The count byte holds that count less one, +the way a token states one. A token playing the phrase at that count names the phrase alone and spends a +byte fewer. -A song's phrases come from two places: the samples it plays, each offering the planes it -writes, and a search over whatever those leave uncovered. Both are weighed the same way — -a phrase keeps its entry by sparing the streams more bytes than the entry costs. +A song's phrases come from two places: the samples it plays, each offering the planes it writes, and a +search over whatever those leave uncovered. Both are weighed the same way. A phrase keeps its entry when it +saves the streams more bytes than the entry costs. ### B.4 The token streams -Each plane is a byte sequence written as tokens. The opcode's top two bits name the kind -and the low six carry its operand: +Each plane is a byte sequence of tokens. The opcode's top two bits name the kind and the low six carry +its operand: ``` -00cccccc hold the value the plane reached, for c+1 ticks -01nnnnnn b0..bn the n+1 bytes that follow, one per tick -10pppppp cccccccc phrase p, for c+1 ticks -11pppppp cccccccc tt phrase p, for c+1 ticks, every value plus tt +00cccccc hold the value the plane reached, for c+1 symbols +01nnnnnn b0..bn the n+1 symbols that follow, one each +10dppppp cccccccc phrase p, for c+1 symbols +11dppppp cccccccc tt phrase p, for c+1 symbols, every value plus tt ``` -`p == $3F` escapes: the phrase's id is the byte that follows, which reaches every id in -the table while the low ones stay a byte cheaper. The shift `tt` is **added within the -byte**, wrapping — one addition on the 6502, and the same one the encoder agrees with. +`p == $1F` is an escape: the phrase's id is the byte that follows. That reaches every id in the table +while the low ids stay a byte cheaper. The shift `tt` is **added within the byte** and wraps. That is one +addition on the 6502, and the encoder does the same addition. + +**The `d` bit says the count is the phrase's own.** A token carrying it names the phrase and no count, and +plays the count the table states for that phrase. The count byte `cccccccc` is then absent. That bit is +what leaves five bits for an id, so a phrase opcode names ids up to `$1F` outright where a hold or a +literal counts to `$3F`. + +**A token's count is a duration, and it may run past the phrase.** Past its last value the plane holds +that value, so a note whose envelope has finished keeps sounding. A count shorter than the body cuts the +note off. One entry therefore serves every length a figure is played at and, with the shift, every pitch. -**A token's count is a duration, and it may run past the phrase.** Past its last value the -plane holds that value onward, which is how a note whose envelope has finished keeps -sounding; a count short of the body cuts the note off. One entry therefore serves every -length a figure is played at, and — with the shift — every pitch. +**A token counts symbols.** On a plane whose byte carries a repeat count (§C) one symbol +covers as many ticks as it states, so a token's reach in ticks is the ticks its symbols +carry between them. On every other plane a symbol is a tick. ### B.5 Where a song comes round -A song that repeats re-enters its streams partway through, so the tick it returns to -begins a token on every plane, and that token names its values outright rather than -leaning on the value the plane had reached. A bend plane is re-entered at the value its -channel's flags have reached by that tick (§C), which begins a token of its own. Coming -round is then a matter of pointing each plane at the byte the header states and clearing -what it was playing. +A song that repeats re-enters its streams partway through. The tick it returns to therefore begins a +symbol, and a token, on every plane, and that token names its values outright instead of relying on the +value the plane had reached. A bend plane is re-entered at the value its channel's flags have reached by +that tick (section C), which begins a token of its own. Coming round then means pointing each plane at the +byte the header names and clearing what it was playing. -The tick a song returns to is the export's choice: its first tick, the first tick of an -order frame, or none at all, in which case the header states `$FFFF` and the song stops at -its end. +The tick a song returns to is the export's choice: its first tick, the first tick of an order frame, or +none. With none, the header has `$FFFF` and the song stops at its end. ## C. What the planes hold @@ -156,66 +155,76 @@ The planes are written in this order, and each group belongs to one channel: | Plane | Carries | Reaches | |---|---|---| -| pulse 1 control | duty cycle and volume | `$4000` | +| pulse 1 control | duty cycle and volume, under a repeat count | `$4000` | | pulse 1 value | pitch index, and a flag for a bent tick | `$4002`, `$4003` | | pulse 1 bend | divider offset of each flagged tick | `$4002`, `$4003` | -| pulse 2 control | duty cycle and volume | `$4004` | +| pulse 2 control | duty cycle and volume, under a repeat count | `$4004` | | pulse 2 value | pitch index, and a flag for a bent tick | `$4006`, `$4007` | | pulse 2 bend | divider offset of each flagged tick | `$4006`, `$4007` | -| triangle control | linear counter | `$4008` | -| triangle value | pitch index, and a flag for a bent tick | `$400A`, `$400B` | +| triangle value | pitch index, a flag for a bent tick, and the index that rests | `$4008`, `$400A`, `$400B` | | triangle bend | divider offset of each flagged tick | `$400A`, `$400B` | -| noise control | volume | `$400C` | -| noise value | period and mode | `$400E` | - -Splitting a channel's registers apart is what gives each plane something to repeat: a -volume envelope and a pitch line are separate series that turn over at their own rates. - -**Every channel's planes are in every block.** A channel an export leaves out holds its -silent values from the first tick to the last, which a hold covers in a few bytes, so the -driver reads the same layout whichever channels a song sounds. - -**A plane playing zero throughout is absent.** Every plane starts at zero, so a plane that -never leaves it takes no stream: both its header entries state `ABSENT_STREAM` (`$FFFF`), -and the driver leaves it standing at zero on every tick. A tone channel that never bends -costs its bend plane nothing this way. - -**The noise channel reads no bend.** It selects one of sixteen fixed periods, so there is -no finer grid for a bend to reach, and the plane it would hold is left out of the block. - -**A timer's high half reaches the register only where it differs from the last one -written.** Storing it restarts a pulse waveform and reloads the triangle's counter, so a -channel holding one pitch across a rest keeps its phase running the way a rendered channel -does. - -**A value plane names a pitch, not a divider.** Stating a note as its distance above the -lowest one the song reaches is what lets `TRANSPOSED_PHRASE` move a whole phrase by adding -to it, and a divider offset added to an index means nothing. A tone channel's **bend** -plane is where the offset goes: signed bytes in two's complement, added to the divider the -value plane's note resolves to. - -**A bend plane holds a value only where a note bends.** A value byte's low seven bits index -the pitch table and its top bit, `BEND_FLAG`, says the tick reads its offset from the bend -plane; an unflagged tick sounds its pitch's own divider. The bend plane holds one value per -flagged tick, in order, and the driver advances it on those ticks alone. The encoder flags -each note from its first bent tick to its last, so a vibrato passing through zero keeps the -value plane still, and a channel that never bends holds an empty bend plane — an absent one. -The flag sits above every index, so a transposed phrase keeps it. - -A bent frame sounds the divider its note's own is moved to, and **the value plane names -the frame's own note** while the bend plane holds the steps from that note's divider. A row -transposes a note and keeps its bend's steps, so a bent figure keeps the same bend bytes -at every pitch it is played at, which is what lets one dictionary entry serve it. A bend -past the signed byte is counted from the pitch lying nearest the divider instead, a -divider halfway between two pitches going to the higher one; those steps stay inside half -the widest gap between neighboring pitches — 57 at the default tuning — so every divider -the register holds reaches the planes. - -The driver sign-extends that byte and adds it across both halves of the timer, which is the -only arithmetic it performs on a song's behalf. Everything that makes the sum land in -range is settled in Python: `bent_timer` keeps the divider within `[MIN_TIMER, MAX_TIMER]`, -the rule the generators render a bent frame by, so the timer's high half never exceeds -three bits, never reaches the length-counter field beside them, and never collides with the +| noise control | volume, under a repeat count | `$400C` | +| noise value | period and mode, around a repeat count | `$400E` | + +Splitting a channel's registers apart gives each plane something to repeat: a volume envelope and a pitch +line are separate series that turn over at their own rates. + +**Every channel's planes are in every block.** A channel an export leaves out has its silent values from +the first tick to the last, which a hold covers in a few bytes. The driver therefore reads the same layout +whichever channels a song sounds. + +**A plane counts its repeats in the bits its register ignores.** A pulse control byte holds a duty cycle +and a volume around two bits the hardware wants set. A noise control byte holds a volume under the same +two. A noise period byte holds a mode bit above three the register reads nothing from. Those spare bits +carry the ticks the value repeats for, so a run costs one byte however long it lasts. The register byte is +the symbol masked and ored with the bits the hardware fixes, and counting one repeat off subtracts +`COUNT_STEP`. Every other plane spends its whole byte and reads a symbol a tick. The division follows from +the plane, so the block states none of it. + +**A plane standing at the value it is seeded to throughout is absent.** The driver seeds every plane +before its first token: to the bits its register fixes where it has any, and to the index that rests on +the triangle's value plane. A plane that never leaves that value takes no stream, and both its header +entries are `ABSENT_STREAM` (`$FFFF`). A tone channel that never bends spends nothing on its bend plane, a +pulse or noise channel that never sounds spends nothing on its control plane, and a song without a +triangle spends nothing on its value plane. + +**The triangle names its silence in the pitch it plays.** The channel sounds at one level, so a tick +states whether it sounds through the index its value plane names. The index standing above every pitch the +table covers silences the linear counter, and the divider stays where the channel last sounded. That is a +whole plane the block does without. + +**The noise channel has no bend plane.** It selects one of sixteen fixed periods, so there is no finer +grid for a bend to reach, and its block leaves the plane out. + +**A timer's high half reaches the register only where it differs from the last one written.** Storing it +restarts a pulse waveform and reloads the triangle's counter. A channel holding one pitch across a rest +therefore keeps its phase running, as a rendered channel does. + +**A value plane names a pitch, not a divider.** A note is written as its distance above the lowest one the +song reaches. That lets `TRANSPOSED_PHRASE` move a whole phrase by adding to it, and a divider offset +added to an index would mean nothing. A tone channel's **bend** plane carries the offset: signed bytes in +two's complement, added to the divider the value plane's note resolves to. + +**A bend plane has a value only where a note bends.** A value byte's low seven bits index the pitch table. +Its top bit, `BEND_FLAG`, says the tick reads its offset from the bend plane, and an unflagged tick sounds +its pitch's own divider. The bend plane has one value per flagged tick, in order, and the driver advances +it on those ticks alone. The encoder flags each note from its first bent tick to its last, so a vibrato +passing through zero keeps the value plane still. A channel that never bends has an empty bend plane, +which is an absent one. The flag sits above every index, so a transposed phrase keeps it. + +A bent frame sounds the divider its note's own divider is moved to. **The value plane names the frame's +own note**, and the bend plane holds the steps from that note's divider. A row transposes a note and keeps +its bend's steps, so a bent figure has the same bend bytes at every pitch it is played at, and one +dictionary entry serves it. A bend past the signed byte is counted from the pitch lying nearest the +divider, and a divider halfway between two pitches goes to the higher one. Those steps stay inside half +the widest gap between neighboring pitches, 57 at the default tuning, so every divider the register holds +reaches the planes. + +The driver sign-extends that byte and adds it across both halves of the timer. That addition, and the +repeat a symbol counts down, are the whole of the arithmetic it performs on a song's behalf. Python does +everything that makes the sum land in range: `bent_timer` keeps the divider within +`[MIN_TIMER, MAX_TIMER]`, the rule the generators render a bent frame by. The timer's high half therefore +never exceeds three bits, never reaches the length-counter field beside them, and never collides with the `$FF` the driver marks an unwritten shadow by. ## D. Limits @@ -225,11 +234,14 @@ three bits, never reaches the length-counter field beside them, and never collid | Program area | 32 KB from `$8000`, less the driver | | Song length | as many ticks as the streams fit in | | Offsets within the block | `uint16` | -| Ticks one hold or literal covers | up to `MAX_HOLD_TICKS` / `MAX_LITERAL_BYTES` | -| Ticks one phrase token covers | up to `MAX_PHRASE_TICKS` | -| Values one phrase holds | up to `MAX_PHRASE_LENGTH` | -| Phrases one dictionary holds | up to `MAX_PHRASE_IDS` | - -A song reaching past the space behind the driver, or past what an offset field states, -raises `SongTooLargeError` naming the part that overflowed. A project offering more -phrases than a dictionary holds keeps the ones sparing the streams most, and says so. +| Symbols one hold covers | 1–64 (`MAX_HOLD_TICKS`) | +| Symbols one literal covers | 1–64 (`MAX_LITERAL_BYTES`) | +| Symbols one phrase token covers | 1–256 (`MAX_PHRASE_TICKS`) | +| Ticks one symbol covers | 1 to the plane's own count, 16 at the widest | +| Values one phrase holds | up to 255 (`MAX_PHRASE_LENGTH`) | +| Phrases one dictionary holds | up to 255 (`MAX_PHRASE_IDS`) | +| Phrase ids a token's opcode names | up to 31 (`CHEAP_PHRASE_IDS`) | + +A song that reaches past the space behind the driver, or past what an offset field can hold, raises +`SongTooLargeError` naming the part that overflowed. A project that offers more phrases than a dictionary +holds keeps the ones that spare the streams most, and says so. diff --git a/docs/formats/projects.md b/docs/formats/projects.md index d099a8e34..e4ac15330 100644 --- a/docs/formats/projects.md +++ b/docs/formats/projects.md @@ -1,10 +1,9 @@ # Projects -A project gathers a set of voices and arranges them into a song, saved as a single -`.stp` file. It is what the sequencer works with, and what you hand over when you -share a whole piece. See [Project](../concepts/project.md) for what a project is; -this page documents the file. [Reconstructions](reconstructions.md) documents the -converted audio a sample stands on. +A project gathers a set of voices and arranges them into a song, saved as a single `.stp` file. The +sequencer works with it, and you hand it over to share a whole piece. See +[Project](../concepts/project.md) for what a project is. This page documents the file. +[Reconstructions](reconstructions.md) documents the converted audio a sample stands on. ## Structure @@ -13,11 +12,11 @@ A `.stp` file is a zip archive with two kinds of member: * **`project.json`** — the project document (below). * **`reconstructions/.stn`** — one [reconstruction](reconstructions.md) per sample, stored as its own `.stn` member and referenced from the document by its - id. + id. The archive deflates its members, so a member has the reconstruction's payload as it stands. -Keeping the reconstructions in separate members lets `project.json` stay small -while the larger audio data travels alongside it in the same archive. A -[instrument](../glossary.md#instrument) carries no audio, so the document holds it whole. +The reconstructions sit in separate members, so `project.json` stays small and the larger audio data +travels beside it in the same archive. An [instrument](../glossary.md#instrument) has no audio, so the +document holds it whole. ### `project.json` @@ -39,39 +38,35 @@ Every voice carries an `id` and a `name`. The `kind` says what else it carries: | `sample` | the `reconstruction_id` of its audio member | | `instrument` | its `envelopes` — `volume`, `arpeggio` and `duty_cycle` — and the `initial_pitch` and `initial_period` those values are measured against | -Each envelope holds its `items`, one per tick, and the `loop_point` those items repeat -from while a note is held, or `null` where they play once and the last item stands for as -long as the note sounds. Every envelope states its own point, so a two-item duty cycle -circles on its own period beside a longer volume envelope. +Each envelope has its `items`, one per tick, and a `loop_point`: the item they repeat from while a note +is held. It is `null` where the items play once and the last item holds for as long as the note sounds. +See [loop point](../glossary.md#loop-point). ### `song` The arrangement across the four channels: -* `rows_per_pattern` — the row count every pattern in the song shares; -* `order` — the arrangement itself: an ordered list of frames, each frame mapping - every channel to the pattern index it plays, or empty for a silent slot; -* `channels` — per channel, the `name` of the channel it drives and its pool of - `patterns`, each pattern a list of rows. A row - states the `command` its note column holds — the `voice_id` to start, or a - note-off — along with its `transpose` and `volume`. The channel a voice sounds on - is the one whose pool holds the row. +| Field | Contents | +| --- | --- | +| `rows_per_pattern` | the row count every pattern in the song shares | +| `order` | an ordered list of frames, each mapping every channel to the pattern index it plays, or empty for a silent slot | +| `channels` | per channel, the `name` of the channel it drives and its pool of `patterns`, each pattern a list of rows | + +A row has the `command` its note column holds (the `voice_id` to start, or a note-off), its `transpose` +and its `volume`. The channel a voice sounds on is the one whose pool has the row. ## Detached reconstructions -The reconstructions inside a project are -[detached](reconstructions.md#detached-reconstructions) from their original -source-audio paths, so a project stays portable — it carries everything it needs -and no path that would only mean something on the author's machine. +The reconstructions inside a project are [detached](reconstructions.md#detached-reconstructions) from +their original source-audio paths. A project therefore stays portable: it has everything it needs and no +path that means something only on the author's machine. ## Versioning -`project.json` records the project format version it was written with. On load, -_SampleToNES_ requires that version to match the one it supports and declines an -incompatible file rather than misreading it. A file written at a version the -upgrade chain reaches is migrated in memory to the current shape before -deserialization (see -[Data compatibility](../development/release/compatibility.md)). Unknown or extra fields -within a matching version are ignored, which leaves room for the format to grow. +`project.json` records the project format version it was written with. A file at the supported version +loads as it stands. A file at an older version the upgrade chain reaches is migrated in memory to the +current shape first. Any other file is declined. [Data compatibility](../development/release/compatibility.md) +describes the chain. Unknown or extra fields within a matching version are ignored, which leaves room +for the format to grow. The current format version is 1.1. diff --git a/docs/formats/reconstructions.md b/docs/formats/reconstructions.md index fa2fad77d..b6f9402db 100644 --- a/docs/formats/reconstructions.md +++ b/docs/formats/reconstructions.md @@ -3,72 +3,70 @@ A reconstruction is one converted audio sample: the per-channel instruction streams that produce the NES [approximation](../glossary.md#approximation) of an original recording. It is stored as a `.stn` file. [Reconstruction algorithms](../concepts/reconstruction.md) explains -how one is produced; this page documents the file. +how one is produced. This page documents the file. -The file states what is played rather than what it sounds like: the audio is rendered from the -instructions whenever it is needed, which keeps a file to a few megabytes and keeps what the -application shows in step with what an export plays. +The file has what is played, not what it sounds like. The audio is rendered from the instructions +whenever it is needed. That keeps the file small and keeps what the application shows in step with what +an export plays. ## Contents -A `.stn` file holds: - -* **metadata** — the application name and version, and the reconstruction - data-version used to check compatibility on load (see [Versioning](#versioning)); -* **id** — a unique identifier for the reconstruction; -* **configuration** — a frozen snapshot of the - [generation configuration](../guide/configuration.md) used, so the file records - exactly how it was made: sample rate, NES frequency, spectrum method, gamma, - and the rest; -* **coefficient** — the [working level](../glossary.md#working-level-coefficient), - the single scale factor applied to the input so its loudness fit the NES - channels' range. Storing it lets the reconstruction and the original be shown - and played on a common scale; -* **per-channel instructions** — the instruction stream each channel plays, one - [instruction](../glossary.md#instruction) per frame. This is the data a - FamiTracker export is built from. A reconstruction holds a stream for every one - of the four channels (`pulse1`, `pulse2`, `triangle`, `noise`), and a stream of - no frames is a channel standing by: it is written by no export and costs - nothing, while staying open to edit, so writing an envelope into it puts the - channel in play and clearing every envelope takes it out again; -* **per-channel reference pitch** — the note each channel's arpeggio offsets are - measured against, chosen once when the reconstruction is built and stored with - the instructions it describes. An export reads the offsets against this pitch, - so editing an arpeggio moves the frames around a base that stays put (see - [FamiTracker export](famitracker.md)); -* **per-channel held dimensions** — the envelopes each channel leaves to the - player. An instruction states a value for every dimension of its frame, so this - is what says which of them the instrument itself writes; the rest are the - channel's, and the player keeps the value it already holds for them. A channel - in play writes them all as it is built, and clearing an envelope in the - instruments panel adds that dimension here; -* **stems assignment** — the stems setup the reconstruction was built under, the - recording behind each entry, and, per channel, the source holding each frame - (`stems_data`). Every reconstruction carries one: a conversion from a single file - records one stem covering every channel it plays. A frame whose channel is silent - records the resting stem id, `-1`: a frame no source took, where a source's count - of channels at once or a hierarchy left it free, and a frame the decoding settled - on a silent instruction. A frame the reader wrote by hand records the authored - stem id, `-2`, which answers to no recording and stands through every removal; -* **source audio** — per entry, the recording's name and the file it was read from, - in the order the stems setup lists them. The name belongs to the document and the - file to this machine, so a [detached](#detached-reconstructions) reconstruction - keeps every name and states no location. - -A channel standing by rests at a reference pitch of its own, so the first envelope -written into it sounds on a mid-range note, and it leaves every dimension it offers -to the player, which is the record a channel edited down to empty envelopes reaches -as well. A file naming a stream for the channels it plays alone reads as the whole -four, with the rest coming back standing by. - -The stems setup is also what `sampletones convert --stems` reads, written as JSON with the -same fields: one entry per recording, in the order the recordings are given, each naming the -channels it may occupy, the ones it bends, the `drives` it pushes each of them at and the -`channel_cap` channels it may sound at once; and a hierarchy listing the stem ids by -precedence level. An entry stating no `drives` is read at unit drive on every channel it -holds, and one stating no `channel_cap` may sound all four. Two recordings, the first on the -pulses with its second pulse pushed harder and held to one channel a frame, the second on -the rest as it stands: +A `.stn` file is one MessagePack map: + +| Field | Contents | +| --- | --- | +| `metadata` | the application name and version, and the reconstruction data version checked on load (see [Versioning](#versioning)) | +| `id` | a unique identifier for the reconstruction | +| `config` | a frozen snapshot of the [generation configuration](../guide/configuration.md) it was made with: sample rate, NES frequency, spectrum method, gamma, and the rest | +| `coefficient` | the [working level](../glossary.md#working-level-coefficient), the single scale factor applied to the input to fit the NES channels' range. It lets the reconstruction and the original be shown and played on a common scale | +| `instructions_data` | one entry per channel (below) | +| `stems_data` | the stems setup, the recordings behind it, and which stem holds each frame (below) | + +### `instructions_data` + +One entry per channel: + +| Field | Contents | +| --- | --- | +| `channel_name` | `pulse1`, `pulse2`, `triangle` or `noise` | +| `instructions` | the stream the channel plays, one [instruction](../glossary.md#instruction) per frame. A FamiTracker export is built from this | +| `initial_pitch` | the note the channel's arpeggio offsets are measured against, chosen when the reconstruction is built. An export reads the offsets against this pitch, so editing an arpeggio moves the frames around a fixed base (see [FamiTracker export](famitracker.md)) | +| `held_features` | the dimensions the channel governs. The instrument writes the others itself. An export leaves the governed dimensions empty, and the player keeps the value it already has for them | + +A stream of no frames is a channel **standing by**. No export writes it and it costs nothing, and it +stays open to edit. Writing an envelope into it puts the channel in play, and clearing every envelope +takes it out again. A channel standing by has a reference pitch of its own, so the first envelope written +into it sounds on a mid-range note. It leaves every dimension to the player, as a channel edited down to +empty envelopes does. A file that has streams only for the channels it plays reads as all four channels, +and the rest come back standing by. + +### `stems_data` + +| Field | Contents | +| --- | --- | +| `config` | the stems setup the conversion ran under: one entry per recording, and the hierarchy of levels | +| `sources` | one per entry: the `stem_id` it was converted as, the `name` it is known by, and the `path` it was read from, absent once [detached](#detached-reconstructions) | +| `assignments` | per channel, the `stem_ids` holding each frame, parallel to that channel's stream | + +Every reconstruction has this record. A conversion from a single file records one stem covering every +channel it plays. Two ids name no recording. `-1` is a resting frame. `-2` is a frame the reader wrote by +hand: it answers to no recording and survives every removal. A frame rests where its channel is silent: +where no source took it, where a source's channel count or the hierarchy left it free, or where decoding +settled on a silent instruction. + +A recording's name belongs to the document and its path to this machine. A detached reconstruction +therefore keeps every name and has no location. + +### The stems setup as JSON + +`sampletones convert --stems` reads the same setup, written as JSON with the same fields. It has one +entry per recording, in the order the recordings are given. Each entry names the channels it may occupy, +the ones it bends, the `drives` it pushes each of them at and the `channel_cap` channels it may sound at +once. A hierarchy lists the stem ids by precedence level. A drive settles which instruction each frame +records while the conversion runs, so a reconstruction plays the instructions it names. An entry without +`drives` is read at unit drive on every channel it holds, and an entry without `channel_cap` may sound +all four. In this example, the first recording is on the pulses, with its second pulse pushed harder and +held to one channel a frame. The second recording is on the rest as it stands: ```json { @@ -90,31 +88,29 @@ the rest as it stands: ## Detached reconstructions -A reconstruction normally remembers the file each of its recordings was read from. -Embedding one in a [project](projects.md) makes it part of a shareable artifact, -where an absolute path on the author's machine means nothing to anyone else. -Detaching lets those locations go while keeping the instructions, the stems -assignment and every recording's name, so the reconstruction stays self-contained, -a saved project stays portable, and the document still says which recordings it -was built from. +A reconstruction normally remembers the file each of its recordings was read from. Embedding one in a +[project](projects.md) makes it part of a shareable artifact, where an absolute path on the author's +machine means nothing to anyone else. Detaching lets those locations go and keeps the instructions, the +stems assignment and every recording's name. The reconstruction stays self-contained, a saved project +stays portable, and the document still says which recordings it was built from. ## Versioning -Each file records the reconstruction data-version it was written with. On load, -_SampleToNES_ requires that version to match the one it supports and declines a -file written by an incompatible version rather than misreading it. A file -written at a version the upgrade chain reaches is migrated in memory to the -current shape before deserialization (see -[Data compatibility](../development/release/compatibility.md)); the application version -is stored alongside the data version, for reference. +Each file records the reconstruction data version it was written with, and the application version +beside it for reference. A file at the supported version loads as it stands. A file at an older version +the upgrade chain reaches is migrated in memory to the current shape first. Any other file is declined. +[Data compatibility](../development/release/compatibility.md) describes the chain. The current data version is 2.2. ## Storage and export -`.stn` files live in the documents folder. They are binary -([MessagePack](https://msgpack.org/)) and self-contained: everything needed to -play a reconstruction is the instructions, the stems assignment and the frozen -configuration the file carries. The instruction streams can be exported to a -tracker — one instrument per channel, or a whole module — as described in -[FamiTracker export](famitracker.md) and [Bitphase export](bitphase.md). +`.stn` files live in the documents folder. A file is a deflated [MessagePack](https://msgpack.org/) +payload and is self-contained: the instructions, the stems assignment and the frozen configuration are +everything needed to play it. The payload names every field of every frame, and a reconstruction has one +frame per channel per frame of audio. The names repeat for every frame, so they deflate to a small +fraction of the file. A payload stored plainly opens as it stands, so a file written by an earlier build +still opens. + +The instruction streams can be exported to a tracker, as one instrument per channel or as a whole module. +See [FamiTracker export](famitracker.md) and [Bitphase export](bitphase.md). diff --git a/docs/glossary.md b/docs/glossary.md index fb0eeb9de..729f15349 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -1,212 +1,277 @@ # Glossary -Recurring terms used across the documentation, grouped by area. Other documents -link here rather than redefining a term in place. +Short definitions of the terms used across the documentation, grouped by area. Other pages link here +instead of explaining a term again. ## NES sound hardware ### 2A03 (APU) -The NES's sound chip (Ricoh 2A03). Its audio portion, the APU (Audio Processing -Unit), generates all of the console's sound. _SampleToNES_ emulates its four -melodic/percussive channels and no sampled-audio (DPCM) playback. +The NES's sound chip (Ricoh 2A03). Its audio part, the APU (Audio Processing Unit), makes all of the +console's sound. _SampleToNES_ emulates its four melodic and percussive channels. DPCM sample playback is +outside its scope. ### Channel -One of the 2A03's four sound-producing units: `pulse1`, `pulse2`, -`triangle`, and `noise`. The word *generator* names a distinct concept: -the oscillator kinds an instruction library covers (`pulse`, `triangle`, -`noise`) and the classes that implement them. +One of the 2A03's four sound units: `pulse1`, `pulse2`, `triangle` and `noise`. A *generator* is a +different thing: the kind of oscillator an instruction library covers (`pulse`, `triangle`, `noise`), +and the classes that implement it. ### Pulse (square) -A channel that plays a square wave with a selectable duty cycle and 15 volume -levels. The chip has two of them (`pulse1`, `pulse2`). +A channel that plays a square wave. Its duty cycle is selectable and it has 15 volume levels. The chip +has two independent pulse channels: `pulse1` and `pulse2`. ### Triangle -A channel that plays a triangle wave of fixed shape and fixed volume; only its -pitch varies. Its timer divides the APU clock by 32 while the pulse channels divide -by 16, and all three read the same period table, so a triangle note sounds an octave -below the note it is written as: a triangle instruction of pitch P sounds at pitch -P−12. FamiTracker uses the same convention, so an exported note cell plays at the -pitch _SampleToNES_ played it. +A channel that plays a triangle wave of fixed shape and volume. Only its pitch varies. Its timer divides +the APU clock by 32 where the pulse timers divide by 16, and all three read the same period table. A +triangle note therefore sounds an octave below the pulse note with the same period, so a triangle +instruction of pitch P sounds at pitch P−12. FamiTracker uses the same convention, so an exported note +plays at the pitch _SampleToNES_ played it. ### Noise -A channel that plays pseudo-random noise from an LFSR, with 16 period settings, -15 volume levels, and a short/long mode. The period setting divides the APU clock -into the LFSR's shift rate, `APU_CLOCK / NOISE_PERIODS[index]`, spanning 440.0 Hz at -index 0 to 447443.2 Hz at index 15. +A channel that plays pseudo-random noise from an [LFSR](#lfsr). It has 16 period settings, 15 volume +levels and a short and a long mode. ### Duty cycle -The fraction of each period a pulse wave stays high (one of four settings). It -sets the pulse channel's timbre. +The fraction of each period a pulse wave stays high, in one of four settings. It sets the pulse channel's +timbre. + +### Divider + +The number a tone channel counts down from. It sets the pitch, and a smaller divider gives a higher note. ### LFSR -*Linear-feedback shift register* — the circuit that produces the noise channel's -pseudo-random pattern. Its short/long mode changes the pattern's length, and so its -character: long mode repeats every 32767 shifts and reads as noise, short mode -repeats every 93 and turns a high period setting into an audible tone — index 15 -sounds at 447443.2 ÷ 93 ≈ 4811 Hz. Short mode also carries a strong DC asymmetry, its -output bit set 17.2% of the time against long mode's 50%, and that bias is what gives -it its metallic timbre. +*Linear-feedback shift register*: the circuit that makes the noise channel's pseudo-random pattern. Its +mode sets the pattern's length and so its character. Long mode repeats every 32767 shifts and sounds like +noise. Short mode repeats every 93 shifts, so a high period setting sounds as a tone of metallic timbre. ### NES frequency -How many times per second a program updates the channels — for example 60 Hz on -NTSC or 50 Hz on PAL. _SampleToNES_ supports 15–300 Hz, and this rate sets the -reconstruction's frame rate. +How many times per second a program updates the channels, for example 60 Hz on NTSC or 50 Hz on PAL. +_SampleToNES_ accepts 15–300 Hz, and the rate sets the reconstruction's frame rate. ### NTSC / PAL -The two console video standards. Their refresh rates (about 60 Hz and 50 Hz) are -the two most common NES frequencies. +The two console video standards. Their refresh rates, about 60 Hz and 50 Hz, are the most common NES +frequencies. ## Reconstruction ### Reconstruction -The result of approximating an audio sample with the NES channels: one -instruction stream per channel plus the rendered audio. Saved as a `.stn` file. -See [Reconstruction algorithms](concepts/reconstruction.md). +The result of approximating an audio sample with the NES channels: one instruction stream per channel +plus the rendered audio. Saved as a `.stn` file. See +[Reconstruction algorithms](concepts/reconstruction.md). ### Instruction -A single command to one channel for one frame — on or off, *pitch*, *volume*, *duty -cycle* or *noise period*. It is the unit the reconstruction chooses per frame. +A command to one channel for one frame: on or off, *pitch*, *volume*, *duty cycle* or *noise period*. +The reconstruction picks one per channel for each frame. ### Frame -A short, fixed-length slice of the input audio. Within a frame, each channel -holds one instruction. A frame's length is the sample rate divided by the NES -frequency. +A short slice of the input audio. All frames have the same length: the sample rate divided by the NES +frequency. Each channel plays one instruction in a frame. The sequencer also uses the word for one +position in the [order](#order): the patterns the song plays at that point. + +### Tick + +One update of the channels, at the NES frequency. A tick lasts as long as a frame. Envelopes advance one +item per tick, and a tracker row lasts one or more ticks. + +### Stem + +One recording that goes into a reconstruction. Every conversion is a stems conversion. A single file is +one stem that takes every channel it is given, and several recordings share the channels between them. A +reconstruction records which channels each stem was given and which frames it played, so you can hear, +edit or remove one stem on its own. See [Stems reconstruction](concepts/stems.md) and +[Converting audio](guide/converting.md). + +### Level (stems) + +The rank a stem takes when the channels are shared out. A stem on level 1 is offered channels before one +on level 2, so a lead part can take the channels it needs before a background part does. + +### Hierarchy + +The precedence order of the levels in a stems setup. The levels pick channels in that order. See +[Stems reconstruction](concepts/stems.md). + +### Drive + +How hard a stem pushes a channel. It is set per channel when a conversion is set up. `1.00` is the level +the recording was measured at. A higher drive reaches for a louder match, which suits a part that sits +quietly under the others. ### Instruction library -A precomputed catalog holding, for every possible instruction, the waveform -its channel produces and that waveform's spectrum. The search draws its -candidates from the library. Saved as an `.ins` file. See +A precomputed catalog of every possible instruction, with the waveform its channel produces and that +waveform's spectrum. The search draws its candidates from the library. Saved as an `.ins` file. See [Instruction libraries](formats/instruction-libraries.md). ### Approximation -The mixed, rendered audio a reconstruction produces — the NES channels' closest -match to the original sample. +The mixed, rendered audio a reconstruction produces: the NES channels' closest match to the original +sample. ### Working level (coefficient) -A single scale factor applied to the input so its typical frame plays at the level -one NES channel renders at full volume. +A single scale factor applied to the input, so its typical frame plays at the level one NES channel +renders at full volume. ## Analysis and scoring ### Spectrum (feature, histogram) -A frame's frequency content — the representation matching compares, rather than -the raw waveform (two sounds that sound identical can have very different -waveforms). +A frame's frequency content. Matching compares spectra instead of raw waveforms, because two sounds that +sound alike can have very different waveforms. ### FFT / log-FFT / CQT -Three ways to compute a frame's spectrum, trading time resolution against -frequency resolution. CQT (the constant-Q transform) resolves low pitches finely -and is the default. See [Reconstruction algorithms](concepts/reconstruction.md). +Three ways to compute a frame's spectrum. They trade time resolution against frequency resolution. CQT +(the constant-Q transform) resolves low pitches finely and is the default. See +[Reconstruction algorithms](concepts/reconstruction.md). ### Gamma -A setting from 0 to 100 that reshapes the spectrum before comparison: 0 keeps -the raw power spectrum, 100 makes it logarithmic, and values in between -interpolate. Higher gamma emphasizes quiet detail relative to loud peaks. +A setting from 0 to 100 that reshapes the spectrum before comparison. 0 keeps the raw power spectrum, 100 +makes it logarithmic, and values between blend the two. Higher gamma emphasizes quiet detail over loud +peaks. ### Criterion -The score that rates how well a candidate instruction matches a target frame. It -blends a spectral term (frequency shape) with a temporal term (waveform shape). +The score that rates how well a candidate instruction matches a target frame. It blends a spectral term +(frequency shape) with a temporal term (waveform shape). ### β-divergence -The default per-bin spectral distance inside the criterion — a -Kullback–Leibler-style measure of how far one spectrum is from another. +The default per-bin spectral distance in the criterion. It is a Kullback–Leibler-style measure of how far +one spectrum is from another. ### ERB / K-weighting -Perceptual weightings applied so each frequency bin counts in proportion to how -the ear hears it: ERB spaces bins by auditory critical bands, and K-weighting -applies a loudness curve. +Perceptual weightings that make each frequency bin count as much as the ear hears it. ERB spaces bins by +auditory critical bands. K-weighting applies a loudness curve. + +### Column + +The candidates one channel may sound in a frame, best first. The decoder reads the columns into one +candidate per frame. + +### Mix + +The combined sound of the picks a stem already holds in a frame. A new candidate is scored as it would +sound beside the mix. + +### Pick + +The choice of one candidate for one channel in a frame. A frame is assigned pick by pick for as long as a +pick lowers the frame's cost. + +### Resting + +The state of a channel that no stem holds in a frame. It plays its null instruction, which keeps every +channel's stream in step with the frames. + +### Standing by + +The state of a channel whose stream has no frame. No export writes it and it costs nothing, and it stays +open to edit. See [Reconstructions](formats/reconstructions.md#instructions_data). ### Decoder -The strategy that reads a channel's per-frame candidates into the stream it plays, -named by `generation.decoder.selector`. The **greedy** decoder plays each frame's -best candidate; the **Viterbi** decoder (the default) favors continuity, changing a -channel only when the gain in match quality outweighs the cost of the change. +The strategy that reads a channel's per-frame candidates into the stream it plays. The setting is +`generation.decoder.selector`. The **greedy** decoder plays each frame's best candidate. The **Viterbi** +decoder, the default, favors continuity: it changes a channel only when the gain in match quality +outweighs the cost of the change. ### Calibration -A repeatable experiment that tunes the criterion's settings by reconstructing a -fixed test set and scoring the results. See [Calibration](tools/calibration.md). +A repeatable experiment that tunes the criterion's settings by reconstructing a fixed test set and +scoring the results. See [Calibration](tools/calibration.md). + +### Referee / corpus / render / variant + +Terms from calibration. A *referee* is an independent audio-distance judge that scores a reconstruction +against its original. The *corpus* is the fixed set of synthetic test sounds every configuration is run +against. A *render* is one reconstruction of a corpus sound, written as an audio file for listening. A +*variant* is one configuration a run measured. + +## Song compression -### Referee / corpus +### Plane -Terms from calibration: a *referee* is an independent audio-distance judge that -scores a reconstruction against its original; the *corpus* is the fixed set of -synthetic test sounds every configuration is run against; a *render* is one -reconstruction of a corpus sound, written as a WAV file for listening. +One register of one channel across the whole song, stored as one byte per tick. Each plane is a series of +its own, such as a volume envelope or a pitch line. See [Song compression](concepts/compression.md). + +### Token + +The unit a plane is written in. A token is a hold, a literal or a phrase, and the driver reads tokens +forward one tick at a time. + +### Phrase + +A run of values a plane plays, stored once in the song's dictionary and named by tokens wherever it +occurs. Its position in the dictionary is its id. ## Tracker and export ### FamiTracker -A [_tracker application_](http://famitracker.com/) for composing music for the -NES 2A03. _SampleToNES_ exports instruments and modules that it (and its forks) -can load. +A [_tracker application_](http://famitracker.com/) for composing music for the NES 2A03. _SampleToNES_ +exports instruments and modules that it and its forks can load. ### Bitphase -A [_web tracker_](https://github.com/paator/bitphase) whose chips include the NES -2A03. _SampleToNES_ exports documents and instrument presets it can load. See -[Bitphase export](formats/bitphase.md). +A [_web tracker_](https://bitphase.app/) whose chips include the NES 2A03. _SampleToNES_ +exports documents and instrument presets it can load. See [Bitphase export](formats/bitphase.md). ### Tracker / sequencer -A pattern-based music editor. _SampleToNES_'s built-in sequencer arranges -reconstructed samples into a song. +A pattern-based music editor. _SampleToNES_'s built-in sequencer arranges reconstructed samples into a +song. -### Sequence +### Sequence (envelope) -In a FamiTracker instrument, a per-tick envelope for one dimension: volume, -arpeggio, pitch, hi-pitch, or duty/noise mode. +A per-tick list of values that one dimension of a sound follows while a note is held. The dimensions are +volume, arpeggio, pitch, hi-pitch, and duty or noise mode. An **arpeggio** sequence steps the note itself +up and down. The **pitch** and **hi-pitch** sequences bend it in fine and coarse steps (see +[Bend](#bend)). A FamiTracker instrument has one sequence per dimension, and _SampleToNES_ edits the same +shapes. ### Bend -How far a frame sounds from the note it names, counted in steps of the divider the -channel loads. The **pitch** dimension counts one step per item and the **hi-pitch** -dimension sixteen, and the two add up. What a step is worth follows the note: well under -a cent at the lowest notes, widening to a whole semitone at the highest, where the -divider grid is already coarser than the note grid. Only the pulse and triangle channels -read a bend; the noise channel's sixteen periods have no finer grid. +How far a frame sounds from the note it names, counted in steps of the channel's [divider](#divider). +The **pitch** sequence counts one step per item and the **hi-pitch** sequence sixteen, and the two add +up. A step is well under a cent at the lowest notes and widens to a whole semitone at the highest, where +the divider grid is already coarser than the note grid. Only the pulse and triangle channels read a bend. +The noise channel's 16 periods have no finer grid. + +### Row + +One line of a pattern. It says what each channel starts at that moment, and lasts one or more ticks. ### Pattern -A block of tracker rows spanning the channels. A song plays its patterns in an -order. +A block of tracker rows spanning the channels. A song plays its patterns in an order. ### Metric highlight -The row grouping a song is counted in. The **first highlight** is the beat — the -rows one beat spans — and the **second highlight** is the bar that gathers beats. -The tracker tints the row that opens each, and the beat is what a tempo counts: +The row grouping a song is counted in. The **first highlight** is the beat: the number of rows one beat +spans. The **second highlight** is the bar, which gathers beats. The tracker tints the row that opens +each. A tempo counts beats: `beats_per_minute = 60 × nes_frequency / (ticks_per_row × first_highlight)`. ### Groove -The engine ticks each row of a pattern lasts. An engine holds a row for a whole -number of ticks, so a tempo landing between two counts is played by varying the -count from row to row, and the meter places the longer rows on the bar, then the -beat, then inside the beat. Playback reads the groove by the row's position in the +The number of ticks each row of a pattern lasts. A row lasts a whole number of ticks, so a tempo between +two counts is played by varying the count from row to row. The meter places the longer rows on the bar +first, then on the beat, then inside the beat. Playback reads the groove by the row's position in the pattern, so the pattern's first row starts it afresh. ### Order @@ -215,60 +280,57 @@ The list that arranges patterns into the song's timeline. ### Module -A complete FamiTracker song, saved as an `.ftm` file — its settings, -instruments, patterns, and order together. +A complete FamiTracker song, saved as an `.ftm` file: its settings, instruments, patterns and order +together. ### Document -A complete Bitphase project, saved as a `.btp` file — its songs, instruments, -tables, patterns, and order together. +A complete Bitphase project, saved as a `.btp` file: its songs, instruments, tables, patterns and order +together. ### Table -In Bitphase, a per-tick list of semitone offsets a pattern cell attaches to a -channel, which carries the pitch contour a FamiTracker arpeggio sequence would. +In Bitphase, a per-tick list of semitone offsets that a pattern cell attaches to a channel. It carries +the pitch contour a FamiTracker arpeggio sequence would. ### Voice -Anything a tracker row can name: a **sample** or an **instrument**. A project holds its -voices in one list, and a row states which one to start and the step it plays at. +Anything a tracker row can name: a **sample** or an **instrument**. A project keeps its voices in one +list. A row says which voice to start and the step it plays at. ### Sample (sequencer) -A reconstruction added to the sequencer as a playable voice, carrying the -instruction stream its conversion found for each channel. +A reconstruction added to the sequencer as a playable voice. It carries the instruction stream its +conversion found for each channel. ### Sample column -The tracker's leftmost data column. It places a sample across every channel that -sample's reconstruction covers and clears the rest of the row, which is why it takes -samples alone: an instrument sounds on the one channel that names it. It summarizes -what those channels hold, reading `?` where they disagree. See -[The sequencer](guide/sequencer.md#writing-a-pattern). +The tracker's leftmost data column. It places a sample across every channel the sample's reconstruction +covers, and clears the rest of the row. It takes samples only, because an instrument sounds on the one +channel that names it. Its cell summarizes what those channels hold and reads `?` where they disagree. +See [The sequencer](guide/sequencer.md#writing-a-pattern). ### Instrument -One set of envelopes a channel reads while a note sounds, saved as an `.fti` file. A -voice written by hand is a single instrument, placed on whichever channel suits it — -the way a FamiTracker instrument is; a sample carries one instrument per channel it -plays. See [The sequencer](guide/sequencer.md) and -[FamiTracker export](formats/famitracker.md). Bitphase takes the same envelopes as a -`.json` instrument preset. See [Bitphase export](formats/bitphase.md). +One set of envelopes a channel reads while a note sounds, saved as an `.fti` file. A hand-written voice is +a single instrument that goes on whichever channel suits it, as in FamiTracker. A sample has one +instrument per channel it plays. Bitphase takes the same envelopes as a `.json` instrument preset. See +[The sequencer](guide/sequencer.md), [FamiTracker export](formats/famitracker.md) and +[Bitphase export](formats/bitphase.md). ### Initial pitch -The value an instrument's frames are built at, and the note an exported preset is -tuned to. An instrument written by hand states one for the tonal channels and a period -for the noise channel, so the same envelopes sound on any of the four; the note it -actually sounds at comes from the row that places it. The matching value on a sample is -its per-channel [reference pitch](formats/reconstructions.md#contents). +The value an instrument's frames are built at, and the note an exported preset is tuned to. A +hand-written instrument has one for the tonal channels and a period for the noise channel, so the same +envelopes sound on any of the four channels. The row that places the instrument sets the note it sounds +at. A sample's matching value is its per-channel [reference pitch](formats/reconstructions.md#contents). ### Loop point -The item a single envelope repeats from while a note is held, which lets an attack be -followed by a sustained tail. Each envelope states its own, so a two-item duty cycle -circles on its own period beside a longer volume envelope. An envelope without one -holds its last item for as long as the note sounds. +The item a single envelope repeats from while a note is held. It lets an attack be followed by a +sustained tail. Each envelope has its own loop point, so a two-item duty cycle can circle on its own +period beside a longer volume envelope. An envelope without one holds its last item while the note +sounds. ## File types diff --git a/docs/guide/command-line.md b/docs/guide/command-line.md index 97aece3ab..465f0ded9 100644 --- a/docs/guide/command-line.md +++ b/docs/guide/command-line.md @@ -29,10 +29,11 @@ How you run the command depends on how you installed _SampleToNES_: Add `--help` to any command to see its options. -`sampletones --help` also lists commands for developing _SampleToNES_, such as `check` and -`codec`. They run from a copy of the source code, and [Tooling](../development/tooling.md) -describes them. `sampletones calibration` measures how well the app reconstructs a set of reference -sounds; [Calibration](../tools/calibration.md) explains how to run it and read the results. +`sampletones --help` also lists commands for developing _SampleToNES_, such as `check` and `codec`. +They run from a copy of the source code. Add `--help` to one to see what it does. + +`sampletones calibration` measures how well the app reconstructs a set of reference sounds. +[Calibration](../tools/calibration.md) explains how to run it and read the results. ## Options @@ -43,7 +44,8 @@ sounds; [Calibration](../tools/calibration.md) explains how to run it and read t reconstruction goes to the reconstructions folder of your configuration. - `--channels ` sets the channels `convert` may use, for example `--channels pulse1,pulse2`. Without it, `convert` uses pulse 1, triangle and noise. -- `--stems ` gives `convert` a stems file, which it needs to mix several recordings. +- `--stems ` gives `convert` a stems file, which it needs to mix several recordings. The file has + one [stem](../glossary.md#stem) entry per recording. A `convert` command takes either `--channels` or `--stems`. @@ -53,10 +55,9 @@ A `convert` command takes either `--channels` or `--stems`. - **Convert a folder**: `sampletones convert path/to/folder`. Every recording in the folder becomes its own reconstruction, saved in your reconstructions folder. - **Mix several recordings into one reconstruction**: - `sampletones convert bass.wav lead.wav --stems stems.json`. The stems file has one entry per - recording, in the same order. Each entry lists the channels the recording may use and the - channels it bends. The command prints which recording uses which entry before it starts. - [Reconstructions](../formats/reconstructions.md) shows the file. + `sampletones convert bass.wav lead.wav --stems stems.json`. The entries follow the order of the + recordings. Each entry lists the channels the recording may use and the channels it + [bends](../glossary.md#bend). [Reconstructions](../formats/reconstructions.md) shows the file. - **Build a library**: `sampletones library --config my-config.json` GPU support is chosen when you install _SampleToNES_. [Installation](installation.md) explains diff --git a/docs/guide/configuration.md b/docs/guide/configuration.md index 4b566f87a..751e71c67 100644 --- a/docs/guide/configuration.md +++ b/docs/guide/configuration.md @@ -1,22 +1,21 @@ # Configuration -_SampleToNES_ converts audio with a **generation configuration**. The configuration sets the sample -rate, the NES frequency, how the audio is analyzed and how sounds are compared. The settings you -change most often are on the **Main** tab. The configuration file has the rest. +_SampleToNES_ converts audio with one set of settings, the **generation configuration**. It covers the +sample rate, the NES frequency, how the audio is analyzed and how sounds are compared. You change the +everyday ones on the **Main** tab. The configuration file has the rest. -You choose the channels each recording uses, how hard it pushes each of them, and how many of -them it sounds at once on the **Main** tab, each time you set up a conversion. See +The channels each recording uses, how hard it pushes them and how many it sounds at once are separate. +You set them for each recording when you set up a conversion. See [settings for one recording](converting.md#settings-for-one-recording). ## Settings on the Main tab Three cards on the **Main** tab have the everyday settings: -- **General settings**: **Normalize audio**, **Quantize audio**, and the **Sample rate** and **NES - frequency** a library is built for. -- **Source settings**: the channels and bends of the recording you selected in the converter's - list, the drive each of those channels is pushed at, and how many of them the recording may - sound at once. +- **General settings**: the **Sample rate** and **NES frequency** a library is built for, + **Normalize audio**, which evens out the loudness of the recording before it is matched, and + **Quantize audio**, which coarsens it to fewer volume steps first. +- **Source settings**: the settings of the recording you selected in the converter's list. - **Advanced settings**: **Method** and **Feature scaling**, which set how the app measures the frequencies in the audio, the number of **Workers**, and the library and output folders. Choose **View ▸ Show advanced settings** to show this card. [Reconstruction @@ -26,16 +25,10 @@ The app saves your changes to `config.json`. See [Where your files live](files.m ## The configuration file -`config.json` has more settings than the **Main** tab shows. For example, you can change: - -- the [selector](../concepts/reconstruction.md), greedy or Viterbi -- the phase aligner -- the scoring weights and the distance measure -- the number of candidates kept for each frame - -The [configuration file reference](../formats/configuration.md) lists every setting. -[Reconstruction algorithms](../concepts/reconstruction.md) explains what each one does, and lists -the defaults. +`config.json` holds the other settings. They control how each frame's sound is analyzed, how candidates +are scored and how each channel's stream is chosen. The [configuration file +reference](../formats/configuration.md) lists every setting, and [reconstruction +algorithms](../concepts/reconstruction.md) explains what each one does and the value it starts at. To load or save a whole configuration, use **Reconstruction ▸ Load generation settings...** and **Reconstruction ▸ Save generation settings...**. On the [command line](command-line.md), add diff --git a/docs/guide/converting.md b/docs/guide/converting.md index 5accce935..659f5c1c4 100644 --- a/docs/guide/converting.md +++ b/docs/guide/converting.md @@ -1,90 +1,118 @@ # Converting audio -The **Main** tab (`F1`) turns audio files into -[reconstructions](../concepts/reconstruction.md). You gather the recordings you -want on the **Converter** card, choose which NES channels each one may use, -decide whether every recording becomes its own reconstruction or they all mix -into one, and start the conversion. +The **Main** tab (`F1`) turns audio files into [reconstructions](../concepts/reconstruction.md). You +gather the recordings you want on the **Converter** card, decide whether each one becomes its own +reconstruction or they all mix into one, choose which NES channels each recording may use, and start +the conversion. ## Choosing what to convert -The **Converter** card lists the recordings a conversion uses. Add them from the **Filesystem** browser: +The **Converter** card lists the recordings a conversion uses. Add them from the **Filesystem** +browser: -- Double-click an audio file, or Ctrl-click it, to add it. You can also right-click it and choose **Add as stem**. -- Ctrl-click a folder, or right-click it and choose **Add folder**, to add every recording inside it, at any depth in the folder tree. +- Double-click an audio file, or Ctrl-click it, to add it. You can also right-click it and choose + **Add as stem**. A [stem](../glossary.md#stem) is one recording that goes into a reconstruction. +- Ctrl-click a folder, or right-click it and choose **Add folder**, to add every recording inside it, + at any depth in the folder tree. -Turn on **Playback ▸ Autoplay** (`Ctrl+P`) to play a recording with a single click. This lets you listen through a folder before adding anything from it. With Autoplay off, right-click a recording and choose **Play**. +Turn on **Playback ▸ Autoplay** (`Ctrl+P`) to play a recording with a single click. This lets you +listen through a folder before adding anything from it. With Autoplay off, right-click a recording and +choose **Play**. -Adding a folder opens a small window while the app reads the folder. **Stop** ends the search and keeps the list as it was. If the folder has no recordings, the window says so and the list stays the same. +A small window shows while the app reads a folder. **Stop** ends the search and keeps the list as it +was. **x** removes a row from the list. Removing a folder removes every recording in it. -Click a row to select it. The **Source settings** card then shows that recording. Right-clicking a row selects it as well. Press `Del` to remove the selected row. Closing a folder that contains the selected recording clears the selection. +Click a row to select it, and the **Source settings** card shows that recording. Right-clicking a row +selects it as well. Press `Del` to remove the selected row. -## Choosing which channels a recording uses - -The NES has four sound channels: **Pulse 1**, **Pulse 2**, **Triangle**, and **Noise**. Every recording in the list has a checkbox for each channel. Check the channels that the recording may use. Press `1` to `4` to switch a channel on or off for the selected row. +## One reconstruction each, or one mix from all -A folder's checkboxes show the channels of all the recordings inside it: +**Output**, at the top of the **Converter** card, sets what the conversion makes: -- checked if all recordings use the channel, -- filled with the channel's color if only some recordings use it, -- empty if none of them use it. +- **One per recording** — each recording in the list becomes its own reconstruction. Recordings added + with a folder are saved in a matching folder structure. +- **One from all** — all recordings are mixed into a single reconstruction. Folders are replaced by + the recordings inside them. -Click the checkbox to change the channel for all recordings in the folder. To change the channel for one recording, open the folder first: click the marker next to the folder name, or double-click the folder name. Each recording inside then has its own checkboxes. +A mix can hold up to eight recordings. If you switch to **One from all** with more than eight +recordings in the list, a dialog asks which ones to mix. The same dialog opens when you add a folder +with more recordings than the mix has room for. Double-click a row in the dialog to hear the +recording. When the mix is full, uncheck a recording before you check another one. -## Settings for one recording +When a mix has two or more recordings, the rows are grouped into [**levels**](../glossary.md#level-stems). +Levels set which recordings get their channels first: recordings on level 1 get channels before +recordings on level 2. +This lets a lead melody take the channels it needs before a background part does. -The **Source settings** card sets how the recording you selected in the list uses its channels. It has one line per channel: +Drag a row onto another row to put them on the same level. Drag it into the gap between levels to give +it a level of its own. You can also right-click a row to use the same commands. -- **on** repeats the checkbox in the list, so you can also switch a channel there. -- **bend** tunes each note to the recording's exact pitch. **Pulse 1**, **Pulse 2**, and **Triangle** have it; noise has none. -- **drive** sets how hard the recording pushes that channel. `1.00` is the calibrated level, and up to `5.00` pushes it harder, which suits a part that sits quietly under the others. Drag the slider, or Ctrl-click it to type a value. - -**Channels at once**, below the lines, sets how many of its channels the recording may sound in a single frame. Set it to 1 to hear the recording on one channel at a time. It never sounds more channels than it uses. - -A folder shows what the recordings inside it agree on and reads **mixed** where they differ. Changing anything settles every recording in the folder on it. +**Order** sets how the levels take turns: -With no row selected, the card reads **New recordings** and holds the settings every recording you add starts with. The app remembers them between sessions. +- **Round robin** — every level gets a turn in each round. +- **Strict** — one level gets all its channels before the next level chooses. -## One reconstruction each, or one mix from all +## Choosing which channels a recording uses -**Output**, at the top of the **Converter** card, sets what the conversion makes: +The NES has four sound [channels](../glossary.md#channel): **Pulse 1**, **Pulse 2**, **Triangle** and +**Noise**. Every recording in the list has a checkbox for each channel. Check the channels that the +recording may use. Press `1` to `4` to switch a channel on or off for the selected row. -- **One per recording** — each recording in the list becomes its own reconstruction. Recordings added with a folder are saved in a matching folder structure. -- **One from all** — all recordings are mixed into a single reconstruction. Folders are replaced by the recordings inside them. +A folder's checkboxes represent every recording inside it. A checkbox is checked when they all use the +channel, partly filled when only some do, and empty when none do. Clicking one changes the channel for +all of them. To change one recording on its own, open the folder first — click the marker +next to the folder name, or double-click the name — and each recording inside has its own checkboxes. -A mix can hold up to eight recordings. If you switch to **One from all** with more than eight recordings in the list, a dialog asks which ones to mix. The same dialog opens when you add a folder with more recordings than the mix has room for. +## Settings for one recording -The dialog lists the same rows as the card and shows how many recordings you selected. Double-click a row to hear the recording. **Add** becomes available when your selection fits. When the mix is full, uncheck a recording before you check another one. +The **Source settings** card sets how the recording you selected uses its channels. It has one line +per channel: -When a mix has two or more recordings, the rows are grouped into **levels**. Levels set which recordings get their channels first. Recordings on level 1 get channels before recordings on level 2. This lets a lead melody take the channels it needs before a background part does. +- **on** repeats the checkbox in the list, so you can also switch a channel there. +- [**bend**](../glossary.md#bend) tunes each note to the recording's exact pitch. **Pulse 1**, **Pulse 2** and **Triangle** + have it, and noise does not. +- [**drive**](../glossary.md#drive) sets how hard the recording pushes that channel. `1.00` is the level the recording was + measured at, and up to `5.00` pushes it harder, which suits a part that sits quietly under the + others. Drag the slider, or Ctrl-click it to type a value. -Drag a row onto another row to put them on the same level. Drag it into the gap between levels to give it a level of its own. You can also right-click a row to use the same commands. +**Channels at once**, below the lines, sets how many of its channels the recording may sound in a +single frame. Set it to 1 to hear the recording on one channel at a time. It never sounds more +channels than it uses. -**Order** sets how the levels take turns: +A folder shows the setting its recordings share, and **mixed** where they differ. Changing a setting +on a folder changes it for every recording inside. -- **Round robin** — every level gets a turn in each round. -- **Strict** — one level gets all its channels before the next level chooses. +With no row selected, the card is called **New recordings**. It has the settings every recording you add +starts with, and the app remembers them between sessions. ## Running a conversion -Click the button under **Output** to start the conversion. The button's label tells you what it is about to do. Only one conversion can run at a time. While a conversion is running, the button reads **Cancel**. - -**Destination:** shows where the result is saved: a single file for one conversion, or a folder for a longer run. Click the path to open it in your file manager. If the conversion would replace an existing reconstruction, the app asks you first. - -When the conversion finishes, click **Load** to open the result on the **Reconstruction** tab, where you can [listen to it and export it](reconstruction.md). After a conversion of several recordings, the button reads **Open** instead. +Click the button under **Output** to start the conversion. Its label says what it is about to do, and +reads **Cancel** while the conversion runs. Only one conversion runs at a time. -The first conversion with new settings builds the [instruction library](../concepts/instruction-library.md) for those settings. This takes a while. Later conversions with the same settings use the same library. +**Destination:** shows where the result is saved: a single file for one conversion, or a folder for a +longer run. Click the path to open it in your file manager. If the conversion would replace an +existing reconstruction, the app asks you first. -## The instruction library +When the conversion finishes, click **Load** to open the result on the **Reconstruction** tab, where +you can [listen to it and export it](reconstruction.md). After a conversion of several recordings, the +button opens the folder instead. -The **Instructions** tab (`F4`) builds and browses the [instruction library](../concepts/instruction-library.md), the catalog of NES tones that a conversion searches. +The first conversion with new settings takes longer, because it builds the [instruction +library](../concepts/instruction-library.md) for them. Later conversions with the same settings reuse +it. -A conversion builds the library it needs by itself, so you rarely need this tab. Use it to build a library before a long session, or to explore the sounds your settings can make. +## Building a library yourself -Select an instruction to see its **Waveform** and **Spectrum**. This lets you see and hear a single NES tone on its own. Click the waveform to play the tone from that point. +The **Instructions** tab (`F4`) builds and browses the [instruction +library](../concepts/instruction-library.md), the catalog of NES tones that a conversion searches. A +conversion builds the library it needs by itself, so you rarely need this tab. Use it to build a +library before a long session, or to explore the sounds your settings can make. -**Generate library** builds a library for your current settings. Once a library is loaded, the button reads **Regenerate instructions**. +**Generate library** builds a library for your current settings. A library marked **[!]** was built by +another version of _SampleToNES_; click it to rebuild it. -A library marked **[!]** was built by another version of _SampleToNES_. Click it to rebuild it. +Select an [instruction](../glossary.md#instruction) to see its **Waveform** and **Spectrum**. This +lets you see and hear a single NES tone on its own. Click the waveform to play the tone from that point. diff --git a/docs/guide/files.md b/docs/guide/files.md index f4bc690c7..30c7f06df 100644 --- a/docs/guide/files.md +++ b/docs/guide/files.md @@ -18,36 +18,36 @@ tab, or right-click in the **Filesystem** browser. ## File types -| Type | What it is | Where it is saved | -| --- | --- | --- | -| `.ins` | [instruction library](../formats/instruction-libraries.md) | `instructions/` | -| `.stn` | [reconstruction](../formats/reconstructions.md) | `reconstructions/` | -| `.stp` | [project](../formats/projects.md) | `projects/` | -| `.fti` | FamiTracker instrument | where you choose | -| `.ftm` | FamiTracker module | where you choose | -| `.json` | Bitphase instrument preset | where you choose | -| `.btp` | Bitphase project | where you choose | -| `.nsf` | NES sound file | where you choose | - -The last five types are exports: - -- Open `.fti` and `.ftm` files in [FamiTracker](../formats/famitracker.md). -- Open `.json` and `.btp` files in [Bitphase](../formats/bitphase.md). -- Play `.nsf` files in an NSF player or on a NES. +| Type | What it is | +| --- | --- | +| `.ins` | [instruction library](../formats/instruction-libraries.md) | +| `.stn` | [reconstruction](../formats/reconstructions.md) | +| `.stp` | [project](../formats/projects.md) | +| `.fti` | [FamiTracker instrument](../formats/famitracker.md) | +| `.ftm` | [FamiTracker module](../formats/famitracker.md) | +| `.json` | [Bitphase instrument preset](../formats/bitphase.md) | +| `.btp` | [Bitphase project](../formats/bitphase.md) | +| `.nsf` | [NSF program](../formats/nsf.md) | + +The first three are your own work, saved in the folders above. The rest are exports, and the save +dialog asks where each one goes. [FamiTracker](../formats/famitracker.md) opens `.fti` and `.ftm`, +[Bitphase](../formats/bitphase.md) opens `.json` and `.btp`, and an `.nsf` plays in an NSF player or +on a NES. ## Naming exported files The save dialog lists the file types that fit your export, and adds the extension of the type you -pick. When you export one channel, the dialog lists all three instrument types, so you can pick the -program there. You can also type the extension yourself. +pick. When you export one channel, the dialog lists all three instrument types, so you can choose the +program the file is for. You can also type the extension yourself. -The name you type also names the instrument in the tracker: +The name you type also names the instrument in the tracker. What the app saves depends on the export: -| Export | The name you type | What the app saves | -| --- | --- | --- | -| **Instruments** panel ▸ **Export instrument...** | the file | one file, and the instrument inside it has the same name | -| **Reconstruction ▸ Export instruments** | the set | one file per channel, each named ` (channel)`. For `.nsf`, one file, and its title is set in the **Export NSF program** window | -| **File ▸ Export** | the file | one file with the whole song | +- **Export instrument...** in the **Instruments** panel saves one file. The name you type is the file's + name, and the instrument inside it has the same name. +- **Reconstruction ▸ Export instruments** saves one file per channel. The name you type is the name of + the set, and each file is named ` (channel)`. An `.nsf` export is one file, and its title is set + in the **Export NSF program** window. +- **File ▸ Export** saves one file with the whole song. For example, exporting a reconstruction named `Kick` to FamiTracker instruments saves `Kick (pulse1).fti`, `Kick (triangle).fti`, and one file for each other channel the reconstruction diff --git a/docs/guide/getting-started.md b/docs/guide/getting-started.md index f3dcd0a7d..1387b52ff 100644 --- a/docs/guide/getting-started.md +++ b/docs/guide/getting-started.md @@ -6,21 +6,19 @@ a song. First, [install](installation.md) _SampleToNES_. ## Reconstruct a sound into FamiTracker instruments 1. Launch the app and open the **Main** tab. -2. In the **Filesystem** browser, double-click an audio file (WAV, MP3, FLAC, - OGG, AIFF, or AU) — or Ctrl-click a folder, to reconstruct every audio file - inside it. -3. If you want, click the recording in the list and check the channels it may use - under **Source settings**, and change **General settings**. Each recording - needs at least one channel. -4. Click the button under **Output** to start the conversion. The first time you - convert with a given set of settings, the [instruction - library](../concepts/instruction-library.md) it needs is built first - ("Generating instructions library..."), which takes a while. -5. When it finishes, click **Load** to open the result on the **Reconstruction** - tab. -6. Choose **Reconstruction ▸ Export instruments ▸ FamiTracker instruments...** and - name the export. The app writes one `.fti` file per instrument: `Kick - (pulse1).fti`, `Kick (triangle).fti`, and so on. +2. In the **Filesystem** browser, double-click an audio file, for example `Kick.wav`. WAV, MP3, FLAC, + OGG, AIFF and AU files work. To reconstruct every audio file in a folder, Ctrl-click the folder + instead. +3. Optional: click the recording in the list and check the channels it may use under **Source + settings**. Each recording needs at least one channel. +4. Optional: change **General settings**, such as the sample rate. +5. Click the button under **Output** to start the conversion. The first conversion with a new set of + settings takes longer, because the [instruction library](../concepts/instruction-library.md) it + needs is built first. +6. When it finishes, click **Load** to open the result on the **Reconstruction** tab. +7. Choose **Reconstruction ▸ Export instruments ▸ FamiTracker instruments...** and name the export + `Kick`. The app writes one `.fti` file per channel: `Kick (pulse1).fti`, `Kick (triangle).fti`, + and so on. That is the shortest path from a sound to instruments you can load in FamiTracker. [Converting audio](converting.md) and [working with a @@ -32,13 +30,14 @@ in full. 1. Choose **File ▸ New project**. The app switches to the **Sequencer** tab. 2. Have one or more reconstructions ready — make them as above, or open existing ones. -3. Add each as a sample: in the Sequencer's **Browser**, right-click a +3. Add each as a [sample](../glossary.md#sample-sequencer): in the Sequencer's **Browser**, right-click a reconstruction and choose **Add to Sequencer**. If its NES frequency differs from the project's, confirm with **Add anyway**. 4. In the **Tracker** grid, click a cell and type notes on your keyboard. To assign a sample to a channel, right-click a cell and choose **Set voice**. -5. Arrange the piece in the **Order** grid, and set **Rows**, **Tempo**, **Speed**, - and **NES frequency** under **Module options**. +5. Arrange the piece in the [**Order**](../glossary.md#order) grid. Under **Module options**, set + **Rows** (the length of a pattern), **Speed** (the [ticks](../glossary.md#tick) each row lasts), + **Tempo** and **NES frequency**. 6. Choose **File ▸ Export ▸ FamiTracker module...** and pick a path for the `.ftm` file. **Bitphase project...** next to it writes the same song as a `.btp`. diff --git a/docs/guide/installation.md b/docs/guide/installation.md index 4e2236393..6100c2fec 100644 --- a/docs/guide/installation.md +++ b/docs/guide/installation.md @@ -1,6 +1,6 @@ # Installation -You can install _SampleToNES_ in three ways: +You can install _SampleToNES_ in several ways. If you are unsure, download a release. - **Download a release.** This is the easiest way on Windows and Linux. - **Install from PyPI.** This works on Windows, macOS and Linux. diff --git a/docs/guide/interface.md b/docs/guide/interface.md index f81e8af86..b1912afe4 100644 --- a/docs/guide/interface.md +++ b/docs/guide/interface.md @@ -5,28 +5,21 @@ _SampleToNES_ has four tabs. Use `F1` to `F4` to switch between them: - [**Main**](converting.md) (`F1`) — turn audio files into [reconstructions](../concepts/reconstruction.md). - [**Reconstruction**](reconstruction.md) (`F2`) — listen to a reconstruction, edit its instruments, and export it. - [**Sequencer**](sequencer.md) (`F3`) — arrange reconstructions into a song. -- [**Instructions**](converting.md#the-instruction-library) (`F4`) — build and browse the [instruction library](../concepts/instruction-library.md) a conversion uses. +- [**Instructions**](converting.md#building-a-library-yourself) (`F4`) — build and browse the [instruction library](../concepts/instruction-library.md) a conversion uses. You usually work through the tabs in this order. Convert your files on **Main**. When the conversion finishes, click **Load** to open the result on **Reconstruction**. From there, **Add to Sequencer** adds the reconstruction to a song. -The **Instructions** tab is optional. Use it to explore single _instructions_: the smallest sounds the NES sound chip makes. +The **Instructions** tab is optional. Use it to explore single [instructions](../glossary.md#instruction), the smallest sounds the chip makes. ## The menus -Each menu covers one kind of work: - -- **File** — projects, exporting a song as a module or a program, and rendering a song to audio. -- **Edit** — undo and redo, followed by commands for whatever you have selected. Its lower half changes with what you are working on. -- **Reconstruction** — creating, opening, and saving reconstructions, and exporting them. -- **Voice** — adding voices to the sequencer, and commands for the voice you selected. -- **Playback** — playing, autoplay, following the song, and muting channels. -- **View** — advanced settings, favorites, the display, and the shortcuts. -- **Help** — **About**. +Each menu covers one kind of work. The lower half of **Edit** acts on what you have selected. **Voice** +acts on the voice you selected. **Reconstruction** acts on the reconstruction you have open. Two items are easy to miss: -- **View ▸ Show advanced settings** shows the **Advanced settings** card on the **Main** tab. It contains the generation method, feature scaling, worker count, and library and output folders. [Configuration](configuration.md) explains each one. -- **Playback ▸ Audio settings...** chooses the device, sample rate, and buffer size you listen through. **Sample rate** and **NES frequency** on the **Main** tab are different settings: they set how the audio is converted. +- **View ▸ Show advanced settings** shows the **Advanced settings** card on the **Main** tab. It has options you rarely need: how the audio is analyzed, the number of workers, and the library and output folders. [Configuration](configuration.md) explains each one. +- **Playback ▸ Audio settings...** chooses the device, sample rate, and buffer size you listen through. These differ from **Sample rate** and **NES frequency** on the **Main** tab, which set how the audio is converted. Project properties belong to a project and are covered in the [sequencer guide](sequencer.md). @@ -34,6 +27,6 @@ Project properties belong to a project and are covered in the [sequencer guide]( **View ▸ Keyboard shortcuts...** (`Ctrl+K`) lists everything you can do from the keyboard and lets you change any shortcut. Click an action's shortcut and press the keys you want. If another action already uses those keys, the app names that action and asks whether to reassign them. -**Reset to defaults** restores the original shortcuts. Your changes take effect when you click **OK**. They are saved with your settings and are still there the next time you start. +**Reset to defaults** restores the original shortcuts. Your changes take effect when you click **OK**, and the app keeps them for the next time you start. `Space` plays and pauses, and `Esc` stops. On macOS, the shortcuts use Command where other platforms use Control. diff --git a/docs/guide/reconstruction.md b/docs/guide/reconstruction.md index 50536f993..4ca27b4a5 100644 --- a/docs/guide/reconstruction.md +++ b/docs/guide/reconstruction.md @@ -1,64 +1,83 @@ # Working with a reconstruction -The **Reconstruction** tab (`F2`) is where you listen to a reconstruction, -compare it against the audio it was made from, edit the instruments it plays, -and export it. Open one from the **Browser**, or click **Load** after -[a conversion](converting.md) on the **Main** tab. +The **Reconstruction** tab (`F2`) is where you listen to a +[reconstruction](../glossary.md#reconstruction), compare it against the audio it was made from, edit +the instruments it plays, and export it. Open one from the **Browser**, or click **Load** after [a +conversion](converting.md) on the **Main** tab. ## Listening to a reconstruction -Open a saved reconstruction from the **Browser** on the **Reconstruction** tab. - -The browser groups reconstructions in two ways: +The **Browser** groups reconstructions in two ways: - **By configuration** groups them by the settings they were made with. - **By sample** groups every version of the same source audio together. -If the reconstruction you have open has unsaved changes, the app asks whether to save it first. +To keep frequently used reconstructions within reach, right-click a reconstruction or a folder and +choose **Mark as favorite**. Check **Favorites only** to show only those items. -To keep frequently used reconstructions within reach, right-click a reconstruction or a folder and choose **Mark as favorite**. Check **Favorites only** to show only those items. +If the reconstruction you have open has unsaved changes, opening another one asks whether to save it +first. -The **Source** card switches playback between **Reconstruction** and **Original audio**, so you can compare the two. The **Waveform** card has a checkbox for each channel. Keys `1` to `4` switch the same checkboxes. +The **Source** card switches playback between **Reconstruction** and **Original audio**, so you can +compare the two. The **Waveform** card has a checkbox for each channel, and keys `1` to `4` switch the +same checkboxes. -Click the waveform to play from that point. While playback is paused, a click moves the playback position. Drag the waveform to move the view, and double-click to fit the view. +Click the waveform to play from that point. While playback is paused, a click moves the playback +position. Drag the waveform to move the view, and double-click to fit the view. ## Hearing what each recording contributed -The **Stems** card lists the recordings used to build a reconstruction. It groups them by the level each one was given. Every row has a checkbox for each channel that the recording used, and the checkbox at the front toggles all of them. - -A colored square at the left of each row is the color that recording is drawn in under the waveform, so you can tell which bars came from which recording. - -Double-click a row to show that recording in your file browser. +The **Stems** card lists the recordings a reconstruction was built from, grouped by the +[level](converting.md#one-reconstruction-each-or-one-mix-from-all) each one was given. A row has a +checkbox for each channel the recording used, and the checkbox at the front switches all of them. +Double-click a row to show the recording in your file browser. -Under the waveform is a row of colored bars, one line per channel. A letter at the left of each line names its channel: **P** for Pulse 1, **p** for Pulse 2, **T** for Triangle and **N** for Noise. Each bar shows which recording played that stretch, in the recording's own color; a dark stretch means nothing was played there. The bars appear when a reconstruction was built from more than one recording. +Uncheck a box to hear the reconstruction without that recording on that channel. The waveform, the +playback, the original audio and a WAV export all follow the checkboxes, so you can hear what each +recording added, channel by channel. The saved reconstruction keeps every channel, whatever you +uncheck. -Uncheck a channel to silence it in the waveform, in playback, in the original audio and in a WAV export. The **Instruments** panel follows too: it shows the envelopes of the part you are listening to, and the sizes it states measure that part. Uncheck every recording on a channel and the channel reads as empty. This lets you see and hear what one recording added, channel by channel. The saved reconstruction keeps every channel, whatever you uncheck. +Each recording has a color. The bars under the waveform use it to show which recording played each +stretch of each channel. -The checkboxes also decide what an instrument edit changes. A frame belongs to the recording that played it, so an edit changes the frames of the recordings you have checked and leaves the rest alone. Uncheck a recording to shape one part of a channel without touching the others. +The **Instruments** panel follows the same checkboxes. It shows only what you have checked, its sizes +measure only that, and an edit changes only that. To reshape one recording's part of a channel, uncheck the others first. -Notes you write into silence belong to no recording. They gather in an **Edits** row below the recordings, with checkboxes of its own, and removing a recording leaves them alone. +Notes you write where no recording played gather in an **Edits** row, which has checkboxes of its own. +Removing a recording leaves them alone. **Collapse levels** shows the whole list as one table. -**x** at the end of a row removes that recording from the reconstruction. The app asks you to confirm first. The recording goes silent, its row disappears, and the change is saved when you save the reconstruction. A reconstruction needs at least one recording, so the **x** of the last row is disabled. The **Edits** row has no **x**, because it names no recording. +**x** at the end of a row removes that recording, after you confirm. The change is saved when you save +the reconstruction. The **x** of the last recording is disabled, because a reconstruction needs at least +one. The **Edits** row has no **x**. ## Editing instruments -The **Instruments** panel shows what each channel plays. You edit the sequences by dragging the bars or typing values. +The **Instruments** panel shows what each channel plays, as one +[sequence](../glossary.md#sequence-envelope) per dimension. A colored band beneath each set of bars +shows which recording each frame came from. Edit a sequence by dragging its bars or typing values. -Each channel has its own set of sequences: +Each channel has its own set: -- **Pulse 1** and **Pulse 2** — volume, arpeggio, pitch, hi-pitch, and duty cycle. -- **Triangle** — volume, arpeggio, pitch, and hi-pitch. -- **Noise** — volume, arpeggio, and duty cycle. +- **Pulse 1** and **Pulse 2** — volume, arpeggio, pitch, hi-pitch and [duty + cycle](../glossary.md#duty-cycle). +- **Triangle** — volume, arpeggio, pitch and hi-pitch. +- **Noise** — volume, arpeggio and duty cycle. -Type `|` before a value to mark where the sequence repeats while a note is held. `15 14 | 12 10` plays the attack once and then loops the last two values. +Type `|` before a value to mark where the sequence repeats while a note is held. `15 14 | 12 10` plays +the attack once and then loops the last two values. -Clear a sequence to use the channel's own setting. For example, an instrument with an empty volume sequence plays at the volume the channel is set to. Each channel shows how many bytes its instrument takes on the NES, so you can see how much space an edit uses. +Clear a sequence to use the channel's own setting. For example, an instrument with an empty volume +sequence plays at the volume the channel is set to. Each channel shows how many bytes its instrument +takes on the NES, so you can see how much space an edit uses. -You can also edit an **instrument** here, a voice you write by hand. The [sequencer guide](sequencer.md#voices-samples-and-instruments) describes instruments. Right-click one in the **Voices** list and choose **Edit**. +You can edit an [**instrument**](../glossary.md#instrument) here as well. It is a voice you write by +hand, described in the [sequencer guide](sequencer.md#voices-samples-and-instruments). Right-click one +in the **Voices** list and choose **Edit**. -For an instrument, the panel shows **Audition** in place of the pitch steppers. Choose **Pulse**, **Triangle** or **Noise**, and the note keys play the instrument on that sound generator. +For an instrument, the panel shows **Audition** in place of the pitch steppers. Choose **Pulse**, +**Triangle** or **Noise**, and the note keys play the instrument on that channel. ## Exporting @@ -66,11 +85,17 @@ You export a reconstruction from the **Reconstruction** menu: - **Export instruments ▸ FamiTracker instruments...** writes one `.fti` file per channel. - **Export instruments ▸ Bitphase presets...** writes the same instruments as `.json`. -- **Export instruments ▸ NSF program...** opens the **Export NSF program** window and writes a single `.nsf` file that plays the whole reconstruction on a NES. The window works as it does [in the sequencer](sequencer.md#exporting-an-nsf-program). A reconstruction has no order frames, so **Repeat** has no **From a frame**. It has no samples either, so **Level** has no **Samples**. You can only select the channels the reconstruction uses. -- **Export to WAV...** renders the audio using the channel and stem checkboxes you have set. - -Every export writes what you see and hear: the instrument files hold the same part of each channel that the **Instruments** panel draws. - -**Export instrument...** in the **Instruments** panel writes only the channel you are looking at, in whichever format you choose in the save dialog. See [where your files live](files.md#naming-exported-files). +- **Export instruments ▸ NSF program...** opens the **Export NSF program** window and writes a single + `.nsf` file that plays the whole reconstruction on a NES. The window matches the one [in the + sequencer](sequencer.md#exporting-the-song), except that **Repeat** has no **From a frame** and + **Level** has no **Samples**. You can select the channels the reconstruction uses. +- **Export to WAV...** renders the audio using the channel and recording checkboxes you have set. + +Every export writes what you see and hear. The instrument files hold the same part of each channel that +the **Instruments** panel shows. + +**Export instrument...** in the **Instruments** panel writes only the channel you are looking at, in +whichever format you choose in the save dialog. See [where your files +live](files.md#naming-exported-files). To use a reconstruction in a song, right-click it and choose **Add to Sequencer**. diff --git a/docs/guide/sequencer.md b/docs/guide/sequencer.md index 520bb775b..6c1f5d2b2 100644 --- a/docs/guide/sequencer.md +++ b/docs/guide/sequencer.md @@ -1,15 +1,15 @@ # The sequencer -The **Sequencer** tab (`F3`) is a tracker. You use it to arrange voices into a song on the four NES -channels. You can export the song as a FamiTracker [module](../formats/famitracker.md) (`.ftm`), -play it back, or render it to audio. +The **Sequencer** tab (`F3`) is a [tracker](../glossary.md#tracker--sequencer). You use it to arrange +voices into a song on the four NES [channels](../glossary.md#channel). You can play the song back, +export it, or render it to audio. The sequencer works on a [project](../formats/projects.md). Choose **File ▸ New project** to start one, or open an existing `.stp` file. ## Voices: samples and instruments -A song plays **voices**. There are two kinds: +A song plays [**voices**](../glossary.md#voice). There are two kinds: - A **sample** is a reconstruction that the song plays. - An **instrument** is a sound you write by hand. Use instruments for melodies and bass lines. @@ -25,62 +25,44 @@ The **Voice** menu adds a voice in four ways: | **Import instrument...** | A FamiTracker instrument file (`.fti`), as an instrument | | **Add to Sequencer** | The reconstruction open on the **Reconstruction** tab, as a sample | -The first three items are also at the top of the **Voices** list. Right-click below the rows of -the list to open the same menu. +The first three items are also at the top of the **Voices** list. Right-click below the rows of the +list to open the same menu. **Add to Sequencer** is on the **Reconstruction** tab as well, and in the +right-click menu of any reconstruction in the **Browser**. -**Add to Sequencer** is also on the **Reconstruction** tab. On the Sequencer tab, right-click a -reconstruction in the **Browser** to find it. +If a reconstruction uses a different [NES frequency](../glossary.md#nes-frequency) than the project, +the app shows **Different NES frequency**. Click **Add anyway** to add it. -If a reconstruction uses a different NES frequency than the project, the app shows **Different -NES frequency**. Click **Add anyway** to add it. +Right-click a voice to rename, duplicate, move or remove it. Some commands do more than their names +say: -To change how a new instrument sounds, right-click it and choose **Edit**. The instrument opens on -the **Reconstruction** tab. See [editing instruments](reconstruction.md#editing-instruments). +- **Edit** opens the voice on the **Reconstruction** tab, where you change how it sounds. See + [editing instruments](reconstruction.md#editing-instruments). +- **New instrument from**, on a sample, copies what the sample plays on one channel into a new + instrument you can edit. +- **Export instrument...** saves the voice as an `.fti` file. A sample has one instrument per + channel, so the app asks which channel to export. -An imported `.fti` file keeps its volume, arpeggio and duty cycle sequences. **Instrument -imported** lists any other settings of the file, which the instrument leaves out. See [reading an instrument -file](../formats/famitracker.md#c-reading-an-instrument-file). +The same commands are on the **Edit** menu for the voice you selected. The right-click menu also +shows how many bytes the voice takes on the NES. This matters when you export an NSF program. -A sample's right-click menu has **New instrument from**. Choose a channel to copy what the sample -plays on that channel into a new instrument you can edit. +Removing a voice that patterns still use asks you first, and clears every row that uses it. -Right-click any voice to use these commands: - -- **Edit**, **Rename**, **Duplicate** and **Remove**. -- Commands that move the voice up or down the list. -- **Export instrument...**, which saves the voice as an `.fti` file. A sample has one instrument - per channel, so the app asks which channel to export. - -The **Edit** menu has the same commands for the voice you selected. - -The right-click menu also shows how many bytes the voice takes on the NES. For a sample, it shows -the total and the size of each channel. - -If patterns still use a voice, removing the voice asks you first. Removing it clears every row -that uses it. - -Hover over a voice to see its name, its kind, its channels and its size. +An imported `.fti` file keeps its volume, arpeggio and duty cycle sequences. **Instrument imported** +lists any other settings the file carried, which the instrument leaves out. See [reading an +instrument file](../formats/famitracker.md#c-reading-an-instrument-file). ## Writing a pattern -The **Tracker** grid is the pattern editor. Each row is one step in time. The grid has a -**Sample** column and a column for each channel: **Pulse 1**, **Pulse 2**, **Triangle** and -**Noise**. Each channel column has a voice, a pitch and a volume. - -Click a cell and type its value. +The **Tracker** grid is the [pattern](../glossary.md#pattern) editor. Each row is one step in time. +The grid has a **Sample** column and a column for each channel: **Pulse 1**, **Pulse 2**, +**Triangle** and **Noise**. Each channel column has a voice, a pitch and a volume. -Right-click a cell for more commands: +Click a cell and type its value. Right-click a cell for the same commands as a menu, including +**Note off**, which stops the note. -- **Set voice** chooses the voice for the cell. -- **Note off** stops the note. -- **Clear cell** and **Clear row** empty the cell or the whole row. -- Transpose and volume commands change the pitch and volume. -- **Play from here** plays from the row you clicked. -- **Play from this frame** plays from the top of the frame on screen. - -The **Sample** column places a sample on every channel the sample uses, and clears the other -channels of the row. The **Sample** column takes samples only. To place an instrument, use the -column of the channel you want it on. +The [**Sample** column](../glossary.md#sample-column) places a sample on every channel the sample +uses, and clears the other channels of the row. It takes samples only. To place an instrument, use +the column of the channel you want it on. A `?` in the **Sample** column means the channels of that row play different voices. @@ -97,24 +79,72 @@ Type notes on your keyboard like a piano: - The bottom two rows of keys play one octave: `Z` `S` `X` `D` `C` and so on. - The two rows above them play the next octave: `Q` `2` `W` `3` `E` and so on. -**Octave**, above the grid, sets the octave of the bottom row. The same keys work in a sample's -cell and type the step that plays the note you pressed. +**Octave**, above the grid, sets the octave of the bottom row. The same keys work in a sample's cell +and type the step that plays the note you pressed. -The noise channel has sixteen sounds in place of notes. Type a noise cell as a signed step, such as -`+03` or `-02`. +The noise channel has sixteen sounds in place of notes. Type a noise cell as a step with a sign, such +as `+03` or `-02`. ## Arranging the song -A song plays patterns in a sequence. The **Order** grid sets that sequence. Each column is one -position in the song. The grid has a row for the **Master** and a row for each channel. +A song plays patterns in a sequence. The [**Order**](../glossary.md#order) grid sets that sequence. +Each column is one position in the song, called a **frame**. The grid has a row for the **Master** +and a row for each channel. -Type a pattern number in the **Order** grid to place a pattern. Right-click a frame for more -commands: +Type a pattern number in the **Order** grid to place a pattern. Right-click a frame to insert, clear +or remove frames, and to repeat one: -- **Duplicate** repeats the frame with the same patterns. +- **Duplicate** repeats the frame with the same patterns, so a change to one shows in both. - **Clone** repeats the frame with new copies of its patterns, so you can change them separately. -- **Insert frame**, **Clear frame** and **Remove** add, empty and delete frames. -- **Play from this frame** plays from that frame. + +## Playing the song + +The play controls under the grid play the song. These keys work anywhere on the tab: + +| Key | Action | +|-----|--------| +| `Space` | Play, or pause and resume | +| `Shift+Space` | Play from the start | +| `Ctrl+Space` | Play from the frame on screen | +| `Ctrl+Shift+Space` | Play from the cursor's row | +| `Esc` | Stop | +| `Ctrl+L` | **Loop song**: start the song again when it ends | + +`Esc` also stops a sample preview. The same commands are on the **Playback** menu, and each grid's +right-click menu plays from the row or frame you clicked. + +## Following the playback + +**Playback ▸ Follow playback** chooses whether the view moves with the song. The app remembers your +choice. + +| Mode | Key | What the view does | +|------|-----|---------------------| +| **Follow rows** | `Ctrl+F` | Scrolls the tracker to the playing row, and shows the playing frame | +| **Follow patterns** | `Ctrl+Shift+F` | Shows the playing frame, and keeps your scroll position | +| **Don't follow** | `Ctrl+Alt+F` | Stays where you put it | + +In every mode, the **Order** grid marks the playing frame, and the tracker marks the playing row. +Use **Follow patterns** or **Don't follow** to type while the song plays. + +## Muting channels + +Click a channel's name at the top of the tracker to mute the channel. Click the name again to unmute +it. The channel names in the **Order** grid work the same way. + +| Gesture | Action | +|---------|--------| +| Click a channel's name | Mute or unmute the channel | +| `Ctrl`+click a channel's name | Solo the channel: silence the rest. `Ctrl`+click again to restore the previous mix | +| Click **Sample** (tracker) or **Master** (order) | Mute or unmute every channel | +| Right-click a name | The same commands as a menu | + +**Playback ▸ Channels** shows which channels play. **Unmute all channels** unmutes all four. Keys +`1` to `4` mute and unmute a channel when the cursor is outside the grids. Inside the grids, the +digit keys type values. + +Muting changes only what you hear. Saving, exporting, rendering and undo use every channel. Opening, +creating or closing a project unmutes all channels. ## Selecting, copying and pasting @@ -135,7 +165,7 @@ selection. | `Shift+Home` / `Shift+End` | Extend the selection to the first or last row (tracker) or position (order) | | `Ctrl+A` | Select the whole frame, or the whole order | | `Ctrl+Shift+A` | Select your column (tracker), or your channel's row (order) | -| `Ctrl+Alt+A` | Select your subcolumn (tracker) | +| `Ctrl+Alt+A` | Select the part of the column the cursor is in (tracker) | | `Ctrl+C` | Copy | | `Ctrl+X` | Cut | | `Ctrl+V` | Paste at the cursor | @@ -151,13 +181,11 @@ copied in the order pastes into the order. A paste starts at the cursor and fills down and to the right: - In the **Tracker**, each cell keeps its kind. A volume pastes into a volume column, wherever you - paste. Cells past the last row or column are dropped. A `?` cell leaves the target cell as it - was. + paste. Cells past the last row or column are dropped. - In the **Order**, a paste past the last frame adds frames to the song. A paste stops at the **Noise** row. -Emptying cells keeps the rows and frames. Every copy, cut, paste and delete is one step in the -history, so one **Undo** reverses it. +Emptying cells keeps the rows and frames. A copy also goes to your clipboard as text. You can paste a block into another open window of _SampleToNES_, or into a message. Voices are copied by their number in the **Voices** list, so in @@ -177,64 +205,21 @@ In the **Tracker**, transpose and volume keys change every cell in the selection With nothing selected, the keys change the cell under the cursor. The same commands are on the right-click menu. -## Playing the song - -The transport under the grid plays the song. These keys work anywhere on the tab: - -| Key | Action | -|-----|--------| -| `Space` | Play, or pause and resume | -| `Shift+Space` | Play from the start | -| `Ctrl+Space` | Play from the frame on screen | -| `Ctrl+Shift+Space` | Play from the cursor's row | -| `Esc` | Stop | -| `Ctrl+L` | **Loop song**: start the song again when it ends | - -`Esc` also stops a sample preview. The same commands are on the **Playback** menu. - -## Following the playback - -**Playback ▸ Follow playback** chooses whether the view moves with the song. The app remembers your -choice. - -| Mode | Key | What the view does | -|------|-----|---------------------| -| **Follow rows** | `Ctrl+F` | Scrolls the tracker to the playing row, and shows the playing frame | -| **Follow patterns** | `Ctrl+Shift+F` | Shows the playing frame, and keeps your scroll position | -| **Don't follow** | `Ctrl+Alt+F` | Stays where you put it | - -In every mode, the **Order** grid marks the playing frame, and the tracker marks the playing row. -Use **Follow patterns** or **Don't follow** to type while the song plays. - -## Muting channels +## Undoing a change -Click a channel's name at the top of the tracker to mute the channel. Click the name again to -unmute it. The channel names in the **Order** grid work the same way. - -| Gesture | Action | -|---------|--------| -| Click a channel's name | Mute or unmute the channel | -| `Ctrl`+click a channel's name | Solo the channel. `Ctrl`+click again to restore the previous mix | -| Click **Sample** (tracker) or **Master** (order) | Mute or unmute every channel | -| Right-click a name | The same commands as a menu | - -**Playback ▸ Channels** shows which channels play. **Unmute all channels** unmutes all four. Keys -`1` to `4` mute and unmute a channel when the cursor is outside the grids. Inside the grids, the -digit keys type values. - -Muting changes only what you hear. Saving, exporting, rendering and undo use every channel. Opening, -creating or closing a project unmutes all channels. +You can undo every change, including a copy, a paste or a delete, which each count as one step. The +**History** panel lists your changes, and you can click one to go back to that point. **Undo** and **Redo** are +on the **Edit** menu. ## Timing and properties -**Module options** sets the song's timing: **Rows** per pattern, **Tempo**, **Speed** and **NES -frequency**. If the project has voices, changing **NES frequency** changes how they play, so the +**Module options** sets the song's timing: **Rows** per pattern, **Speed** (the number of [ticks](../glossary.md#tick) +each row lasts), **Tempo**, and the **NES frequency** the song plays at. Speed and tempo together set how fast +the rows go by. If the project has voices, changing **NES frequency** changes how they play, so the app asks **Change NES frequency** first. Check **Don't ask again** to skip the question. **File ▸ Project properties...** sets the title, the author and the comment. The exported module -includes them. - -**Project properties** also sets the meter: +includes them. It also sets the [meter](../glossary.md#metric-highlight): - **First highlight** is the number of rows in a beat. - **Second highlight** is the number of rows in a bar. @@ -246,32 +231,27 @@ The tempo counts beats, so the meter also changes how fast the song feels. [Temp groove](../formats/bitphase.md#d-tempo-as-a-groove) explains how the app spreads a tempo over the rows. -## Undo and export - -You can undo every change. The **History** panel lists your changes. **Undo** and **Redo** are also -on the **Edit** menu. Click a change in the **History** panel to go back to that point. +## Exporting the song -To export the song: +**File ▸ Export** writes the song in three formats: -- **File ▸ Export ▸ FamiTracker module...** saves an `.ftm` file. See [FamiTracker +- **FamiTracker module...** saves an `.ftm` file. See [FamiTracker export](../formats/famitracker.md) for its contents and limits. -- **File ▸ Export ▸ Bitphase project...** saves a `.btp` file. -- **File ▸ Export ▸ NSF program...** saves an `.nsf` file, which the NES or an NSF player plays - directly. An NSF program has room for 32 KB, so the app tells you when a song is too long. See [NSF - export](../formats/nsf.md) and [song compression](../concepts/compression.md). +- **Bitphase project...** saves a `.btp` file. +- **NSF program...** saves an `.nsf` file, which the NES or an NSF player plays directly. -## Exporting an NSF program +An NSF program has room for 32 KB, so the app tells you when a song is too long. See [NSF +export](../formats/nsf.md) and [song compression](../concepts/compression.md). -**File ▸ Export ▸ NSF program...** opens the **Export NSF program** window. Click **Export** without -changing anything to save the whole song, repeating from the start. +**NSF program...** opens the **Export NSF program** window. Click **Export** without changing +anything to save the whole song, repeating from the start. | Setting | What it does | |---------|--------------| | **Title**, **Artist**, **Copyright** | The text an NSF player shows. Each one holds 31 bytes | | **Channels** | The channels the program plays. A channel you clear stays silent. Select at least one | -| **Repeat** | **Play once** stops at the end. **From the start** plays the song again. **From a frame** goes back to the order frame you type in **Frame**, numbered as in the order list | +| **Repeat** | **Play once** stops at the end. **From the start** plays the song again. **From a frame** goes back to the order frame you type in **Frame** | | **Level** | How much the song is compressed. **None** exports fastest and makes the largest file. **Full search** is the slowest and makes the smallest file | -| **File** | Where the file is saved. **Browse...** opens the save dialog | **Export** closes the window and shows the export's progress. @@ -284,13 +264,12 @@ opens. |---------|--------------| | **Format** | **WAV** for full quality, **MP3** for a smaller file | | **Sample rate** | Samples per second. 44100 Hz is the usual choice | -| **Bit depth** (WAV) | 16-bit PCM is the usual choice. 8-bit sounds rougher, like the NES | -| **Bitrate** (MP3) | A higher bitrate sounds better and makes a larger file. The choices depend on the sample rate | +| **Bit depth** (WAV) | How much detail each sample carries. 16-bit is the usual choice; 8-bit sounds rougher, like the NES | +| **Bitrate** (MP3) | How much data a second of audio takes. A higher bitrate sounds better and makes a larger file. The choices depend on the sample rate | | **Normalize peak** | Makes the song louder until its loudest moment reaches full volume, keeping the balance between channels | -| **File** | Where the file is saved. **Browse...** opens the save dialog. Click the path to open its folder | **Length** shows how long the file will be. Click **Render** to start. **Cancel** stops the render and saves no file. When the render finishes, click the path to open its folder. -A render plays the song once, with every channel, whatever you muted or looped. While a render -runs, conversions and library builds wait, and a render waits for them in the same way. +A render plays the song once, with every channel, whatever you muted or looped. A render, a conversion +and a library build run one at a time, and each waits for the one before it. diff --git a/docs/index.md b/docs/index.md index d88099376..2f1862029 100644 --- a/docs/index.md +++ b/docs/index.md @@ -5,44 +5,41 @@ waves, a triangle and noise. You can arrange the results into a song. You can ex [FamiTracker](glossary.md#famitracker), to [Bitphase](glossary.md#bitphase), or as an `.nsf` program the NES plays. -This documentation explains how to use the app, how it works, and how to build on it. - -The sections below are grouped by what you want to do. They assume different -starting points: the guide needs no prior knowledge, the concepts and formats -sections assume you have used the application, and the API and development -sections are written for programmers. +The sections below are grouped by what you want to do. The guide needs no prior knowledge. The concepts +and formats sections assume you have used the application. The API and development sections are for +programmers. ## Using the application The [**guide**](guide/) walks through the application from installation onward. -- [Installation](guide/installation.md) — a release download, PyPI, running from source, and GPU acceleration. +- [Installation](guide/installation.md) — the ways to install it, and GPU acceleration. - [Getting started](guide/getting-started.md) — your first reconstruction and your first song. -- [The interface](guide/interface.md) — the four tabs, the menus, and the keyboard shortcuts. -- [Converting audio](guide/converting.md) — the Main tab: gathering recordings, choosing channels, and running a conversion. -- [Working with a reconstruction](guide/reconstruction.md) — the Reconstruction tab: listening, editing instruments, and exporting. -- [The sequencer](guide/sequencer.md) — the tracker: arranging samples and hand-written instruments into a song, exporting a module, and rendering it to audio. +- [The interface](guide/interface.md) — the four tabs, the menus and the keyboard shortcuts. +- [Converting audio](guide/converting.md) — the Main tab: adding recordings, choosing channels and running a conversion. +- [Working with a reconstruction](guide/reconstruction.md) — the Reconstruction tab: listening, editing instruments and exporting. +- [The sequencer](guide/sequencer.md) — the tracker: writing a song from samples and instruments, exporting it and rendering it to audio. - [Command line](guide/command-line.md) — running without the graphical interface. - [Where your files live](guide/files.md) — the folders and file types _SampleToNES_ uses. - [Configuration](guide/configuration.md) — the settings you can change, and where. ## How it works -The [**concepts**](concepts/) section explains the ideas behind the -reconstruction. It is written to be read without the source code. +The [**concepts**](concepts/) section explains the ideas behind the reconstruction. You can read it +without the source code. - [Reconstruction algorithms](concepts/reconstruction.md) — how a sample becomes a stream of NES instructions. -- [Stems reconstruction](concepts/stems.md) — how one reconstruction is assigned across several stems. +- [Stems reconstruction](concepts/stems.md) — how the channels are shared between several stems. - [Instruction library](concepts/instruction-library.md) — the catalog of NES sounds the search draws from. -- [Song compression](concepts/compression.md) — how a whole song is fitted into the space an NES program has for it. -- [Project](concepts/project.md) — a whole composition: a song and the reconstructions it is built from. +- [Song compression](concepts/compression.md) — how a song fits into the space an NES program has for it. +- [Project](concepts/project.md) — a song and the reconstructions it is built from. ## Tools -The [**tools**](tools/) section explains the commands that measure _SampleToNES_ or produce -examples: how to run each one with no options, what it writes, and every custom use. +The [**tools**](tools/) section covers the commands that measure _SampleToNES_ or produce examples. +Each page says how to run the command with no options, what it writes and every custom use. -- [Calibration](tools/calibration.md) — how well the reconstruction reproduces a set of reference sounds, with every reconstruction written out to listen to. +- [Calibration](tools/calibration.md) — how well the reconstruction reproduces reference sounds, with every reconstruction written out to listen to. ## File formats @@ -58,33 +55,35 @@ The [**formats**](formats/) section documents the files _SampleToNES_ reads and ## Programming with SampleToNES -The [Python API](api/index.md) covers using `sampletones` as a library, with -worked examples. +The [Python API](api/index.md) page shows how to use `sampletones` as a library, with worked +examples. ## Development -The [**development**](development/) section is for contributors. The documents at its top cover -the whole repository. The documents about the graphical application and about releases have -directories of their own. +The [**development**](development/) section is for contributors. The pages at its top cover the +whole repository. The pages about the graphical application and about releases each have a directory. - [Architecture](development/architecture.md) — the application's layers and the contracts between them. -- [Package layers](development/packages.md) — the packages the repository divides into, and the order they import each other in. -- [Tooling](development/tooling.md) — the `sampletones` command, the tools package and the bootstrap scripts: what each runs on and what it may import. +- [Package layers](development/packages.md) — the packages of the repository, and the order they import each other in. +- [Tooling](development/tooling.md) — the `sampletones` command, the tools package and the bootstrap scripts, with what each runs on and what it may import. - [Coding guidelines](development/guidelines.md) — conventions for the codebase. +- [Writing the documentation](development/documentation.md) — who each document is written for, and how it reads. - [Console player](development/player.md) — the 6502 driver an `.nsf` carries, the codec that fits a song beside it, and how both are verified. -- [Progress](development/progress.md) — how a long operation reports its progress, in one process and across the pool's workers. +- [Progress](development/progress.md) — how a long operation reports progress, within one process and across worker processes. - [Bugs and to-dos](development/bugs-and-todos.md) — the working ledger of known gaps. ### The application -- [Undo engine](development/application/undo.md) — the design of the undo/redo subsystem. +- [Undo engine](development/application/undo.md) — the design of undo and redo. - [Sequencer blocks](development/application/sequencer-blocks.md) — the rules copy, cut, paste and delete follow on both grids. - [Keyboard and actions](development/application/keyboard.md) — how a press reaches behavior, and how an action is declared and shown. - [Identifier vocabularies](development/application/vocabularies.md) — the keys display text is looked up by, and the tags DearPyGui knows a widget by. - [Colors and palettes](development/application/palette.md) — how a color is written, composed, and handed to DearPyGui. - [The render thread](development/application/render-thread.md) — how work reaches DearPyGui from another thread, and what each crossing costs. +- [Dialogs](development/application/dialogs.md) — how a dialog gets its size, and where it opens. - [Playback](development/application/playback.md) — the audio transport shared by every view, and rendering the song to a file. -- [Reconstruction browser](development/application/browser.md) — how a reconstructions directory becomes the tree both browser tabs render, and what narrows it. +- [Reconstruction browser](development/application/browser.md) — how a reconstructions directory becomes the tree both browser tabs show, and what narrows it. +- [Stems in the application](development/application/stems.md) — the Stems card, and what an edit or a removal does to the per-frame record. - [Configuration](development/application/config-organization.md) — how the YAML configuration package is laid out. ### Releases @@ -94,5 +93,5 @@ directories of their own. ## Glossary -The [glossary](glossary.md) defines the recurring terms — NES hardware, the -reconstruction pipeline, and tracker concepts — that the other documents link to. +The [glossary](glossary.md) defines the terms the other pages link to: NES hardware, the reconstruction +pipeline and tracker concepts. diff --git a/docs/tools/calibration.md b/docs/tools/calibration.md index 9d2d118b8..66414df5a 100644 --- a/docs/tools/calibration.md +++ b/docs/tools/calibration.md @@ -6,9 +6,9 @@ writes a page you listen to the results on. Use it to: -- see which spectrum method suits which kind of sound; -- check whether a new version or a changed setting reconstructs better or worse; -- hear what a score means before trusting it. +- See which spectrum method suits which kind of sound. +- Check whether a new version or a changed setting reconstructs better or worse. +- Hear what a score means before trusting it. ## Run it @@ -24,21 +24,27 @@ In an installed copy: sampletones calibration ``` -The run takes several minutes. A first run also builds each instruction library it needs that is -missing or that another version built, which adds to the time. +The run takes a while, because it reconstructs every reference sound once for each variant. A first run +also builds each instruction library it needs, which adds to the time. A library counts as needed when it +is missing or another version built it. With no options, the run measures the program's default settings under the packaged suite: -- every spectrum method: `fft`, `logfft` and `cqt`; -- the channels pulse 1, triangle and noise; -- every other setting at its default value. +- Every spectrum method: `fft`, `logfft` and `cqt`. +- The channels pulse 1, triangle and noise. +- Every other setting at its default value. + +A *variant* is one configuration the run measured, named after the settings that set it apart. For +example, `cqt-pe1` is the `cqt` method at perceptual exponent 1. A *render* is one reference sound as a +variant reconstructed it. The same run on another machine or another version gives figures that compare directly, because the reference sounds are generated from a fixed seed. The results go into a new folder inside the `calibration` folder of your -[SampleToNES folder](../guide/files.md), named by the date and time the run started, for example `run-20260915-124501`. When the run ends, it prints a link to -the report and opens the listening page. +[SampleToNES folder](../guide/files.md). The folder is named by the date and time the run started, for +example `run-20260915-124501`. When the run ends, the command prints a link to the report and opens the +listening page. ## What a run writes @@ -54,27 +60,23 @@ the report and opens the listening page. | `page/` | the stylesheet, the script and the palette the page reads | The audio is FLAC, which is lossless and about a third of the size the same audio takes as WAV. A -default run writes around 35 MB. A run of four channels writes more, because a reconstruction of -four channels has fourteen combinations to cut where one of three has six. - -A *variant* is one configuration the run measured, named after the settings that set it apart, -for example `cqt-pe1` for the `cqt` method at perceptual exponent 1. A *render* is one reference -sound as a variant reconstructed it. +default run writes about 35 MB. A run of four channels writes more, because a reconstruction of four +channels is cut into fourteen combinations and one of three channels into six. ## Listen on the page -Every run writes `index.html` and opens it when it ends. The page holds one row per reference -sound and one column per variant. In each cell: +Every run writes `index.html` and opens it when it ends. The page has one row per reference sound and +one column per variant. Each cell has: -- **play** sounds the whole reconstruction, always from its beginning; -- the number under it is the score, and the line below says how much of the distance between - silence and the recording that reconstruction covers, or **past silence** where silence scores - closer than the reconstruction does; -- one bar per channel shows the frames that channel plays, in the channel's own color. +- **play**, which sounds the whole reconstruction from its beginning. +- The score under it. The line below the score says how much of the distance between silence and the + recording the reconstruction covers, or **past silence** where silence scores closer than the + reconstruction does. +- One bar per channel, showing the frames that channel plays, in the channel's own color. -Beside each bar, **S** plays that channel alone and **M** plays every other channel, so you can -hear what one channel contributes and what the rest sound like without it. Both are separate -recordings the run wrote, so switching between them is exact. +Beside each bar, **S** plays that channel alone and **M** plays every other channel. You hear what one +channel contributes and what the rest sound like without it. Both are separate recordings the run wrote, +so switching between them is exact. Space pauses and resumes, ← and → move across one sound's versions, and ↑ and ↓ move to another sound. @@ -82,30 +84,33 @@ sound's versions, and ↑ and ↓ move to another so The page is drawn in the program's own colors. `--palette NAME` draws it in another of the program's palettes, and `--no-open` leaves it closed and prints its link alone. -A render plays at the same gain as its recording, so a render that sounds quieter is quieter. You -can also play any file in the run folder in an audio player of your own. +A render plays at the same gain as its recording, so a quieter render really is quieter. You can also +play any file in the run folder in an audio player of your own. -Beside each render, a JSON file of the same name holds: +Beside each render, a JSON file of the same name has: -- `judgments`: every referee's score for the render, with the readings behind it; -- `silence`: what complete silence would score against the same recording; +- `judgments`: every referee's score for the render, with the readings behind it. +- `silence`: what complete silence would score against the same recording. - `timelines`: one character per frame for each channel, `1` where that channel plays. A render that scores worse than silence is a sign to listen before trusting the number. ## Read the report -`report.md` holds one section per referee. Each section starts with a table: one row per variant, -one column per category of reference sound, and the overall mean last. Lower is better. +A *referee* is a method that scores a reconstruction against its original: zero for identical signals, +higher for a larger audible difference. [How it judges](#how-it-judges) describes them. + +`report.md` has one section per referee. Each section starts with a table: one row per variant, one +column per category of reference sound, and the overall mean last. Lower is better. The `mr-loudness-dB` section adds a table for each of its readings: - `missing`: content the original has and the reconstruction lacks. A high value sounds dull. - `added`: content the reconstruction brings in. A high value sounds buzzy or noisy. -- `level`: how much louder the reconstruction plays than the original, in decibels. Negative means - quieter. +- `level`: how much louder the reconstruction plays than the original, in decibels. A negative value + means quieter. -`missing` and `added` add up to the score. The level is reported apart from the score. +`missing` and `added` add up to the score. The level is reported apart from it. ## Custom runs @@ -172,32 +177,29 @@ The sounds and their parameters are defined in `sampletones_tools/calibration/co ### The referees -A referee compares a reconstruction with its original and returns a score: zero for identical -signals, higher for a larger audible difference. Referees measure in their own way, apart from the -reconstruction's own scoring, so a comparison stays fair when that scoring is what changed. +Referees measure in their own way, apart from the reconstruction's own scoring, so a comparison stays +fair when that scoring is what changed. Both built-in referees split each signal into bands spaced the way hearing spaces pitch, at several time resolutions, and compare the energy in each band in decibels. Their tuning is in `sampletones_tools/calibration/config/referee.yaml`. - **`mr-auditory-dB`** averages the difference over every band equally. It reads the balance of - tone against noise across the whole spectrum. Because an empty band counts as much as a full one, - a clip that adds noise to a lone tone scores worse than silence. The report still lists this - referee first. + tone against noise across the whole spectrum. An empty band counts as much as a full one, so a clip + that adds noise to a lone tone scores worse than silence. The report lists this referee first for + now (see [Which referee leads](#which-referee-leads)). - **`mr-loudness-dB`** weighs each band by how loud it plays, so the parts you hear carry the score and silent bands barely count. It first brings the reconstruction to the original's level and reports the level difference on its own. Silence scores worst, and a clip with the right tone and some added noise scores between. - **`zimtohrli`** is a model of human hearing from Google. It joins the other two where it is - installed; see [dependencies](../development/release/dependencies.md#calibration). + installed. See [dependencies](../development/release/dependencies.md#calibration). -A referee is tested against sounds whose ranking is known, such as "a triangle at the right pitch -is closer to a sine than silence is". These tests live in -`tests/unit/sampletones_tools/calibration/referee/test_axioms.py`. A test a referee is known to -fail is marked as an expected failure. +A referee is tested against sounds whose ranking is known, such as "a triangle at the right pitch is +closer to a sine than silence is". Rankings it is known to get wrong are recorded as expected failures. ### Which referee leads -The report lists `mr-auditory-dB` first until `mr-loudness-dB` is shown to agree with the ear: rated -by ear, a sweep of renders must rank the way its scores do, with a rank correlation of at least 0.6 -in every category. +The report lists `mr-auditory-dB` first. `mr-loudness-dB` takes the lead once by-ear ratings of a sweep +of renders agree with its scores. [Bugs and to-dos](../development/bugs-and-todos.md) tracks the bar +that agreement must reach. diff --git a/src/sampletones_application/application.py b/src/sampletones_application/application.py index 9397152d7..4befcb549 100644 --- a/src/sampletones_application/application.py +++ b/src/sampletones_application/application.py @@ -1378,8 +1378,7 @@ def content(parent: str) -> None: get_dialog_tag(TAG_GLOBAL_DIALOG_ABOUT), self.language_manager["global.dialog.title.about"], content, - width=about.width, - height=about.height, + geometry=about.window, ) def _refresh_audio_devices(self) -> None: diff --git a/src/sampletones_application/categories/estimate.py b/src/sampletones_application/categories/estimate.py new file mode 100644 index 000000000..7dfed4e5a --- /dev/null +++ b/src/sampletones_application/categories/estimate.py @@ -0,0 +1,26 @@ +from typing import Optional + +from sampletones_application.categories.manager import LanguageManager +from sampletones_shared.utils.time import format_span + + +def time_estimation( + language_manager: LanguageManager, + eta_seconds: Optional[float], +) -> str: + """How long a run has left, phrased to sit beside the count it is reported with. + + A run needs two measurements before it has a rate, so the first moments of one read the mark + the language file keeps for an estimate still being taken, and every later moment reads the + span itself. Both keys are read here, at the moment the line is composed, so a language + chosen mid-run reaches the next reading. + + Args: + language_manager: Where the template and the mark are read from. + eta_seconds: The seconds the run has left, or ``None`` while its rate is still being taken. + + Returns: + str: The clause a status line carries, as ``" (ETA: 2m 13s)"``. + """ + span = language_manager["global.dialog.label.unknown_duration"] if eta_seconds is None else format_span(eta_seconds) + return language_manager["global.dialog.template.time_estimation"].format(eta_string=span) diff --git a/src/sampletones_application/coordinators/tabs/reconstruction.py b/src/sampletones_application/coordinators/tabs/reconstruction.py index b4c6fa55f..e09e3a51c 100644 --- a/src/sampletones_application/coordinators/tabs/reconstruction.py +++ b/src/sampletones_application/coordinators/tabs/reconstruction.py @@ -258,6 +258,7 @@ def __init__( pitch_stepper_style=layout.pitch_stepper_style, copy_width=layout.copy_width, feature_colors=layout.feature_colors, + stem_colors=layout.stem_colors, layout_graphs=layout.graphs, language_manager=language_manager, status_bar=status_bar, diff --git a/src/sampletones_application/layout/general/colors/stem.py b/src/sampletones_application/layout/general/colors/stem.py index 56a967a0f..3debe1df9 100644 --- a/src/sampletones_application/layout/general/colors/stem.py +++ b/src/sampletones_application/layout/general/colors/stem.py @@ -2,6 +2,8 @@ from pydantic import BaseModel +from sampletones_application.utils.palette.colors.base import BaseColor +from sampletones_application.utils.palette.colors.faded import FadedColor from sampletones_application.utils.palette.colors.written import WrittenColor from sampletones_core.constants.algorithm import AUTHORED_STEM_ID, RESTING_STEM_ID @@ -14,6 +16,10 @@ class StemColors(BaseModel, extra="forbid", frozen=True): recording alike. The frames a reader wrote answer to no recording and take a color of their own, and a resting frame shows the ground the ribbon is laid on. + A recording the reader left out keeps that color and carries it faded, so a stretch names its + owner whether or not the reader is listening to it and the reading sits on top of the record. + A rest answers to no recording, so every reader hears it and the ground paints solid. + A conversion may hold more recordings than there are colors, in which case the list starts over, so two recordings far apart on the record can share one color while the neighbors a reader compares stay distinct. @@ -22,24 +28,31 @@ class StemColors(BaseModel, extra="forbid", frozen=True): recordings: Tuple[WrittenColor, ...] authored: WrittenColor rest: WrittenColor + left_out_fraction: float - def for_position(self, position: int) -> WrittenColor: + def for_position(self, position: int) -> BaseColor: """The color the recording standing at ``position`` on the record is known by.""" return self.recordings[position % len(self.recordings)] - def for_stem(self, stem_id: int, position: int) -> WrittenColor: + def for_stem( + self, + stem_id: int, + position: int, + *, + heard: bool, + ) -> BaseColor: """The color one frame's owner is painted in. Args: stem_id: The stem holding the frame. position: Where that stem's entry stands on the record. + heard: Whether the reader hears that stem here. Returns: - WrittenColor: The color the ribbon paints that frame with. + BaseColor: The color the ribbon paints that frame with. """ if stem_id == RESTING_STEM_ID: return self.rest - if stem_id == AUTHORED_STEM_ID: - return self.authored - return self.for_position(position) + solid = self.authored if stem_id == AUTHORED_STEM_ID else self.for_position(position) + return solid if heard else FadedColor(color=solid, fraction=self.left_out_fraction) diff --git a/src/sampletones_application/layout/general/dialogs/about.py b/src/sampletones_application/layout/general/dialogs/about.py index c7cf36dbd..40aaa6dba 100644 --- a/src/sampletones_application/layout/general/dialogs/about.py +++ b/src/sampletones_application/layout/general/dialogs/about.py @@ -1,15 +1,16 @@ from pydantic import BaseModel +from sampletones_application.layout.primitives import DialogGeometry + class AboutDialogLayout(BaseModel, extra="forbid", frozen=True): """The About dialog's size, the size its mark is drawn at, and the room left around the mark.""" - width: int - height: int + window: DialogGeometry logo: int padding: int @property def text_wrap(self) -> int: """Width the text standing beside the mark wraps at.""" - return self.width - self.logo - self.padding + return self.window.width - self.logo - self.padding diff --git a/src/sampletones_application/layout/general/dialogs/dialogs.py b/src/sampletones_application/layout/general/dialogs/dialogs.py index 3d1d8ebf8..945e9fb87 100644 --- a/src/sampletones_application/layout/general/dialogs/dialogs.py +++ b/src/sampletones_application/layout/general/dialogs/dialogs.py @@ -1,15 +1,19 @@ from pydantic import BaseModel from sampletones_application.layout.general.dialogs.about import AboutDialogLayout -from sampletones_application.layout.general.dialogs.height import DialogSizeNoWidth -from sampletones_application.layout.primitives import Dimensions +from sampletones_application.layout.primitives import DialogGeometry class DialogsLayout(BaseModel, extra="forbid", frozen=True): - default: Dimensions - error: Dimensions - recovery: Dimensions - confirmation: DialogSizeNoWidth - text_input: DialogSizeNoWidth - traceback: Dimensions + """The dialogs a reader is answered by, each stating the size it opens at. + + ``traceback_height`` is the height the traceback's text box takes once a reader unfolds it, + which the dialog holding it grows to make room for. + """ + + default: DialogGeometry + error: DialogGeometry + recovery: DialogGeometry + confirmation: DialogGeometry + traceback_height: int about: AboutDialogLayout diff --git a/src/sampletones_application/layout/general/dialogs/height.py b/src/sampletones_application/layout/general/dialogs/height.py deleted file mode 100644 index 2a4bf449b..000000000 --- a/src/sampletones_application/layout/general/dialogs/height.py +++ /dev/null @@ -1,5 +0,0 @@ -from pydantic import BaseModel - - -class DialogSizeNoWidth(BaseModel, extra="forbid", frozen=True): - height: int diff --git a/src/sampletones_application/layout/graphs/bar_plot.py b/src/sampletones_application/layout/graphs/bar_plot.py index ef16c711d..8e568272d 100644 --- a/src/sampletones_application/layout/graphs/bar_plot.py +++ b/src/sampletones_application/layout/graphs/bar_plot.py @@ -11,6 +11,8 @@ class BarPlotLayout(BaseModel, extra="forbid", frozen=True): bar_weight: The share of its slot a bar fills. hover_alpha: How solid the bar under the cursor is drawn. minimum_span: The slots the axis holds room for, so a short dimension keeps a grid. + ownership_band: The share of the value range kept beneath it for the stretches naming + the recording behind each frame. """ min_x: float @@ -19,3 +21,4 @@ class BarPlotLayout(BaseModel, extra="forbid", frozen=True): bar_weight: float hover_alpha: int minimum_span: float + ownership_band: float diff --git a/src/sampletones_application/layout/primitives.py b/src/sampletones_application/layout/primitives.py index 389f5db69..53f653b49 100644 --- a/src/sampletones_application/layout/primitives.py +++ b/src/sampletones_application/layout/primitives.py @@ -1,4 +1,8 @@ -from pydantic import BaseModel +from typing import Final, Optional, Tuple + +from pydantic import BaseModel, Field + +DEARPYGUI_MAXIMUM_WINDOW_SIZE: Final[int] = 30000 class Dimensions(BaseModel, extra="forbid", frozen=True): @@ -6,3 +10,36 @@ class Dimensions(BaseModel, extra="forbid", frozen=True): width: int height: int + + +class DialogGeometry(BaseModel, extra="forbid", frozen=True): + """How large a dialog opens, which is what the place it opens at follows from. + + The width is the dialog's, held both ways: it is the smallest the window may take and the + largest, so every dialog reads at one width whatever it holds and a field, a combo or a + button stretching across the window measures against a width that stands. A window free to + widen to its content and content asking for the window's width feed each other a little + more every frame, which is what holding the width both ways settles. + + The height is the smallest the dialog opens at, and a dialog holding more than that grows + to hold it, so what a reader is shown is always the whole of what the dialog says. A dialog + stating a height opens centered on the frame it is first drawn in, since both numbers stand + before anything is drawn; one stating none is centered against the size it settles at. + """ + + width: int = Field(..., gt=0) + height: Optional[int] = Field(default=None, gt=0) + + @property + def minimum_size(self) -> Tuple[int, int]: + """The size the dialog opens at, which DearPyGui reads as the smallest it may take.""" + return self.width, self.height if self.height is not None else 0 + + @property + def maximum_size(self) -> Tuple[int, int]: + """The size the dialog stops at: the width it states, and as tall as what it holds asks for. + + DearPyGui reads a window's bounds as one pair, so a height left to the content is stated + as the bound DearPyGui carries of its own accord, which is the one no window reaches. + """ + return self.width, DEARPYGUI_MAXIMUM_WINDOW_SIZE diff --git a/src/sampletones_application/layout/project_properties/__init__.py b/src/sampletones_application/layout/project_properties/__init__.py index bbb24c4d6..52e70343e 100644 --- a/src/sampletones_application/layout/project_properties/__init__.py +++ b/src/sampletones_application/layout/project_properties/__init__.py @@ -1,10 +1,10 @@ from pydantic import BaseModel -from sampletones_application.layout.primitives import Dimensions +from sampletones_application.layout.primitives import DialogGeometry class ProjectPropertiesLayout(BaseModel, extra="forbid", frozen=True): - window: Dimensions + window: DialogGeometry label_width: int input_width: int comment_height: int diff --git a/src/sampletones_application/layout/settings/audio.py b/src/sampletones_application/layout/settings/audio.py index ca8eaad9d..7e3152bda 100644 --- a/src/sampletones_application/layout/settings/audio.py +++ b/src/sampletones_application/layout/settings/audio.py @@ -1,9 +1,9 @@ from pydantic import BaseModel -from sampletones_application.layout.primitives import Dimensions +from sampletones_application.layout.primitives import DialogGeometry from sampletones_application.layout.settings.master_gain import MasterGainLayout class AudioSettingsLayout(BaseModel, extra="forbid", frozen=True): - window: Dimensions + window: DialogGeometry master_gain: MasterGainLayout diff --git a/src/sampletones_application/layout/settings/display.py b/src/sampletones_application/layout/settings/display.py index c5ef12341..d47ea8aef 100644 --- a/src/sampletones_application/layout/settings/display.py +++ b/src/sampletones_application/layout/settings/display.py @@ -1,8 +1,8 @@ from pydantic import BaseModel -from sampletones_application.layout.primitives import Dimensions +from sampletones_application.layout.primitives import DialogGeometry class DisplaySettingsLayout(BaseModel, extra="forbid", frozen=True): - window: Dimensions - countdown: Dimensions + window: DialogGeometry + countdown: DialogGeometry diff --git a/src/sampletones_application/layout/settings/export/export.py b/src/sampletones_application/layout/settings/export/export.py index ffdb77547..fbe7ccb6f 100644 --- a/src/sampletones_application/layout/settings/export/export.py +++ b/src/sampletones_application/layout/settings/export/export.py @@ -1,9 +1,9 @@ from pydantic import BaseModel -from sampletones_application.layout.primitives import Dimensions +from sampletones_application.layout.primitives import DialogGeometry from sampletones_application.layout.settings.export.indicator import LoadingIndicatorLayout class ExportSettingsLayout(BaseModel, extra="forbid", frozen=True): - window: Dimensions + window: DialogGeometry indicator: LoadingIndicatorLayout diff --git a/src/sampletones_application/layout/settings/keybindings.py b/src/sampletones_application/layout/settings/keybindings.py index 863f0adfc..0f4a905bb 100644 --- a/src/sampletones_application/layout/settings/keybindings.py +++ b/src/sampletones_application/layout/settings/keybindings.py @@ -1,6 +1,6 @@ from pydantic import BaseModel -from sampletones_application.layout.primitives import Dimensions +from sampletones_application.layout.primitives import DialogGeometry class KeybindingsSettingsLayout(BaseModel, extra="forbid", frozen=True): @@ -10,6 +10,6 @@ class KeybindingsSettingsLayout(BaseModel, extra="forbid", frozen=True): showing, and the action column takes a stated width so every combination reads down one edge. """ - window: Dimensions + window: DialogGeometry list_height: int action_width: int diff --git a/src/sampletones_application/layout/settings/nsf.py b/src/sampletones_application/layout/settings/nsf.py index a3eb7dac9..d918f85fa 100644 --- a/src/sampletones_application/layout/settings/nsf.py +++ b/src/sampletones_application/layout/settings/nsf.py @@ -1,6 +1,6 @@ from pydantic import BaseModel -from sampletones_application.layout.primitives import Dimensions +from sampletones_application.layout.primitives import DialogGeometry class NSFSettingsLayout(BaseModel, extra="forbid", frozen=True): @@ -13,6 +13,6 @@ class NSFSettingsLayout(BaseModel, extra="forbid", frozen=True): frame_width: The width of the field a repeat's order frame is typed into. """ - window: Dimensions + window: DialogGeometry text_width: int frame_width: int diff --git a/src/sampletones_application/layout/settings/render.py b/src/sampletones_application/layout/settings/render.py index 58dc9e84a..aa1fd7114 100644 --- a/src/sampletones_application/layout/settings/render.py +++ b/src/sampletones_application/layout/settings/render.py @@ -1,7 +1,7 @@ from pydantic import BaseModel -from sampletones_application.layout.primitives import Dimensions +from sampletones_application.layout.primitives import DialogGeometry class RenderSettingsLayout(BaseModel, extra="forbid", frozen=True): - window: Dimensions + window: DialogGeometry diff --git a/src/sampletones_application/layout/tabs/main/converter.py b/src/sampletones_application/layout/tabs/main/converter.py index c2428a231..ae3d978f9 100644 --- a/src/sampletones_application/layout/tabs/main/converter.py +++ b/src/sampletones_application/layout/tabs/main/converter.py @@ -1,12 +1,12 @@ from pydantic import BaseModel -from sampletones_application.layout.primitives import Dimensions +from sampletones_application.layout.primitives import DialogGeometry class ConverterLayout(BaseModel, extra="forbid", frozen=True): width: int button_height: int - stem_selection: Dimensions + stem_selection: DialogGeometry stem_selection_footer: int stem_selection_list: int - scan: Dimensions + scan: DialogGeometry diff --git a/src/sampletones_application/logic/instruction/library.py b/src/sampletones_application/logic/instruction/library.py index 86d466490..e6ed8bdc7 100644 --- a/src/sampletones_application/logic/instruction/library.py +++ b/src/sampletones_application/logic/instruction/library.py @@ -2,6 +2,7 @@ from pathlib import Path from typing import Callable, Optional, Tuple +from sampletones_application.categories.estimate import time_estimation from sampletones_application.categories.manager import LanguageManager from sampletones_application.config.managers.config import ConfigManager from sampletones_application.logic.instruction.library_manager import ( @@ -457,18 +458,14 @@ def _update_progress_state(self, task_progress: TaskProgress) -> None: assert self._eta_estimator is not None, "ETA Estimator is not initialized" eta_seconds = self._eta_estimator.update(creator.completed_instructions) - eta_string = ETAEstimator.format_duration(eta_seconds) - status_text = self._language_manager["instructions.library.template.generation_progress_template"].format( creator.completed_instructions, creator.total_instructions, ) - if eta_string: - status_text += self._language_manager["global.dialog.template.time_estimation"].format( - eta_string=eta_string - ) - - self._emit_view(status_text, progress=task_progress.fraction) + self._emit_view( + status_text + time_estimation(self._language_manager, eta_seconds), + progress=task_progress.fraction, + ) def _on_generation_completed(self) -> None: """Closes the generation and reads the catalog again, which lists the library it wrote.""" diff --git a/src/sampletones_application/logic/main/converter/messages.py b/src/sampletones_application/logic/main/converter/messages.py index 239a59840..a01c3d096 100644 --- a/src/sampletones_application/logic/main/converter/messages.py +++ b/src/sampletones_application/logic/main/converter/messages.py @@ -1,12 +1,12 @@ from pathlib import Path from typing import Dict, Final, Tuple +from sampletones_application.categories.estimate import time_estimation from sampletones_application.categories.manager import LanguageManager from sampletones_application.services.conversion.result import ConversionItem from sampletones_application.services.result import ServiceProgress from sampletones_application.view_model.main.converter import ACTIVE_PHASES, ConversionPhase from sampletones_core.library import LibraryState -from sampletones_core.parallelization import ETAEstimator from sampletones_core.reconstructions.stage import ReconstructionStage SINGLE_SOURCE: Final[int] = 1 @@ -33,7 +33,7 @@ def __init__(self, language_manager: LanguageManager) -> None: ReconstructionStage.LOADING: language_manager["main.converter.message.stage_loading"], ReconstructionStage.MATCHING: language_manager["main.converter.message.stage_matching"], ReconstructionStage.DECODING: language_manager["main.converter.message.stage_decoding"], - ReconstructionStage.RENDERING: language_manager["main.converter.message.stage_rendering"], + ReconstructionStage.GATHERING: language_manager["main.converter.message.stage_gathering"], } def progress_text( @@ -48,7 +48,9 @@ def progress_text( batch writes many at once, so a count of the ones written says where it stands. """ return ( - self._run_text(progress, reconstruction_name) + self._stage_text(progress) + self._estimate_text(progress) + self._run_text(progress, reconstruction_name) + + self._stage_text(progress) + + time_estimation(self._language_manager, progress.eta_seconds) ) def preparing_library(self, state: LibraryState) -> str: @@ -112,10 +114,3 @@ def _stage_text(self, progress: ServiceProgress[ConversionItem]) -> str: completed=step.completed, total=step.total, ) - - def _estimate_text(self, progress: ServiceProgress[ConversionItem]) -> str: - eta_string = ETAEstimator.format_duration(progress.eta_seconds) - if not eta_string: - return "" - - return self._language_manager["global.dialog.template.time_estimation"].format(eta_string=eta_string) diff --git a/src/sampletones_application/logic/main/sources/scan.py b/src/sampletones_application/logic/main/sources/scan.py index 56e1328fb..33c12398a 100644 --- a/src/sampletones_application/logic/main/sources/scan.py +++ b/src/sampletones_application/logic/main/sources/scan.py @@ -24,7 +24,8 @@ class FolderScan(CallbackMixin): window keeps answering and the reader knows what it is waiting for. The reports arrive on the worker's own thread, so whoever draws from them crosses to the - thread DearPyGui's context belongs to. + thread DearPyGui's context belongs to. The scan lives in ``logic/`` because it is short and the + Main tab is its only caller. """ def __init__(self) -> None: diff --git a/src/sampletones_application/logic/reconstruction/editing.py b/src/sampletones_application/logic/reconstruction/editing.py index 699e7f20c..fb211c06a 100644 --- a/src/sampletones_application/logic/reconstruction/editing.py +++ b/src/sampletones_application/logic/reconstruction/editing.py @@ -1,19 +1,15 @@ from dataclasses import dataclass -from typing import Dict, Optional, Protocol, Union +from typing import Optional, Protocol, Union -from sampletones_core.constants.enums import ChannelName, FeatureKey +from sampletones_application.view_model.reconstruction.envelopes import ( + ChannelEnvelopesViewModel, +) +from sampletones_core.constants.enums import FeatureKey from sampletones_core.exporters import Features from sampletones_core.features.envelope import Envelope from sampletones_core.project.voices.instrument import Instrument -@dataclass(frozen=True) -class ReconstructionEdit: - """The channels of a loaded reconstruction, each with the envelopes it carries.""" - - channels: Dict[ChannelName, Features] - - @dataclass(frozen=True) class InstrumentEdit: """The one envelope set an instrument carries, as the panel has it in front of a reader. @@ -32,7 +28,7 @@ class InstrumentEdit: features: Features -EditedVoice = Union[ReconstructionEdit, InstrumentEdit] +EditedVoice = Union[ChannelEnvelopesViewModel, InstrumentEdit] class InstrumentEditingProtocol(Protocol): diff --git a/src/sampletones_application/logic/reconstruction/editor.py b/src/sampletones_application/logic/reconstruction/editor.py index fb593d70c..432d66914 100644 --- a/src/sampletones_application/logic/reconstruction/editor.py +++ b/src/sampletones_application/logic/reconstruction/editor.py @@ -6,7 +6,6 @@ from sampletones_application.logic.reconstruction.editing import ( EditedVoice, InstrumentEdit, - ReconstructionEdit, ) from sampletones_application.logic.reconstruction.manager import ReconstructionManager from sampletones_application.view_model.shared.history import HistoryDetail @@ -66,8 +65,7 @@ def edited_instrument(self) -> Optional[EditedVoice]: features=instrument.instrument_features(), ) - feature_data = self._reconstruction_manager.current_features - return None if feature_data is None else ReconstructionEdit(channels=feature_data.channels) + return self._reconstruction_manager.current_features def write_envelope(self, feature_key: FeatureKey, envelope: Envelope[int]) -> None: """Writes one dimension of the instrument in front of the tab, as one history entry. diff --git a/src/sampletones_application/logic/reconstruction/envelopes.py b/src/sampletones_application/logic/reconstruction/envelopes.py new file mode 100644 index 000000000..a0e712f59 --- /dev/null +++ b/src/sampletones_application/logic/reconstruction/envelopes.py @@ -0,0 +1,62 @@ +from typing import Dict + +from sampletones_application.logic.reconstruction.ownership import ownership_lanes +from sampletones_application.view_model.reconstruction.envelopes import ( + ChannelEnvelopesViewModel, +) +from sampletones_application.view_model.shared.ownership import OwnershipLaneViewModel +from sampletones_core.constants.enums import ChannelName +from sampletones_core.exporters import Features +from sampletones_core.reconstructions import Reconstruction +from sampletones_core.reconstructions.reconstruction.stems.selection import StemSelection + + +def heard_envelopes( + reconstruction: Reconstruction, + selection: StemSelection, +) -> ChannelEnvelopesViewModel: + """The envelopes of the part each channel plays for the recordings a reader hears. + + A reconstruction exports one entry per channel whatever it sounds, so a subscript answers + for any of them and :attr:`Features.has_frames` says which ones play. What is drawn, what is + measured and what an export writes are one reading, so a recording switched off on a channel + leaves the envelopes it held there, and a channel every recording is switched off on + describes no frame at all. + + Args: + reconstruction: The reconstruction being read. + selection: The recordings the reader hears, channel by channel. + + Returns: + ChannelEnvelopesViewModel: One entry per channel the reconstruction exports. + """ + channels = { + ChannelName(generator_name): features + for generator_name, features in reconstruction.export_heard(selection).items() + } + return ChannelEnvelopesViewModel( + channels=channels, + ownership=_lanes(reconstruction, selection, channels), + ) + + +def _lanes( + reconstruction: Reconstruction, + selection: StemSelection, + channels: Dict[ChannelName, Features], +) -> Dict[ChannelName, OwnershipLaneViewModel]: + """The stretches under each channel's bars, held to the frames those bars describe. + + Every channel the panel plots takes a lane, whatever the reader has switched on beneath the + waveform: the channel boxes answer for the waveform, the stems card's muted tint and an + export's scope. A lane reaches the last frame the channel's readings describe, so it stands + over the bars drawn from them and no further. + """ + stems_data = reconstruction.stems_data + owned = stems_data.assignments_by_channel + assignments = { + channel_name: owned[channel_name][: features.frame_count] + for channel_name, features in channels.items() + if features.has_frames and channel_name in owned + } + return ownership_lanes(stems_data, assignments, selection.stems_for) diff --git a/src/sampletones_application/logic/reconstruction/feature.py b/src/sampletones_application/logic/reconstruction/feature.py deleted file mode 100644 index 1c5ffe9a1..000000000 --- a/src/sampletones_application/logic/reconstruction/feature.py +++ /dev/null @@ -1,47 +0,0 @@ -from __future__ import annotations - -from dataclasses import dataclass -from typing import Dict - -from sampletones_core.constants.enums import ChannelName -from sampletones_core.exporters import Features -from sampletones_core.reconstructions import Reconstruction -from sampletones_core.reconstructions.reconstruction.stems.selection import StemSelection - - -@dataclass(frozen=True) -class FeatureData: - """The envelopes of every channel a reconstruction holds, keyed by channel. - - A reconstruction exports one entry per channel whatever it sounds, so a subscript answers - for any of them and :attr:`Features.has_frames` says which ones play. The entries answer - for the part the reader is listening to, which is what keeps the plot, the figures and an - export stating one and the same thing. - """ - - channels: Dict[ChannelName, Features] - - def __getitem__(self, channel_name: ChannelName) -> Features: - return self.channels[channel_name] - - @classmethod - def heard(cls, reconstruction: Reconstruction, selection: StemSelection) -> FeatureData: - """The envelopes of the part each channel plays for the recordings a reader hears. - - What is drawn, what is measured and what an export writes are one reading, so a - recording switched off on a channel leaves the envelopes it held there, and a channel - every recording is switched off on describes no frame at all. - - Args: - reconstruction: The reconstruction being read. - selection: The recordings the reader hears, channel by channel. - - Returns: - FeatureData: One entry per channel the reconstruction exports. - """ - return cls( - channels={ - ChannelName(generator_name): features - for generator_name, features in reconstruction.export_heard(selection).items() - } - ) diff --git a/src/sampletones_application/logic/reconstruction/instruments.py b/src/sampletones_application/logic/reconstruction/instruments.py index f6e041abe..194688897 100644 --- a/src/sampletones_application/logic/reconstruction/instruments.py +++ b/src/sampletones_application/logic/reconstruction/instruments.py @@ -7,9 +7,11 @@ from sampletones_application.logic.reconstruction.editing import ( InstrumentEdit, InstrumentEditingProtocol, - ReconstructionEdit, ) from sampletones_application.utils.callbacks.queue import CallbackQueue +from sampletones_application.view_model.reconstruction.envelopes import ( + ChannelEnvelopesViewModel, +) from sampletones_application.view_model.reconstruction.instruments import ( InstrumentViewModel, ReconstructionInstrumentsViewModel, @@ -44,7 +46,7 @@ def __init__( self._pending_reconstruction_update: Optional[ReconstructionUpdate] = None self.on_view_changed: Optional[Callable[[ReconstructionInstrumentsViewModel], None]] = None - self.on_feature_data_changed: Optional[Callable[[Optional[Dict[ChannelName, Features]]], None]] = None + self.on_feature_data_changed: Optional[Callable[[Optional[ChannelEnvelopesViewModel]], None]] = None self.on_reconstruction_instrument_updated: Optional[OnReconstructionInstrumentUpdatedCallback] = None self.on_display_refreshed: Optional[VoidCallback] = None @@ -58,17 +60,20 @@ def update_display(self) -> None: self.call(self.on_feature_data_changed, self._displayed_features()) self.call(self.on_display_refreshed) - def _displayed_features(self) -> Optional[Dict[ChannelName, Features]]: - """The envelopes the panel draws: a reconstruction's channels, or an instrument's own set. + def _displayed_features(self) -> Optional[ChannelEnvelopesViewModel]: + """The envelopes the panel plots: a reconstruction's channels, or an instrument's own set. An instrument is drawn on the tab the panel shows it under, which is the channel offering every - dimension an instrument writes. + dimension an instrument writes, and it answers to no recording, so it carries no stretches. """ instrument = self.instrument_edit if instrument is not None: - return self._instrument_channels(instrument) + return ChannelEnvelopesViewModel( + channels=self._instrument_channels(instrument), + ownership={}, + ) - return self._current_generators() + return self._reconstruction_envelopes() @staticmethod def _instrument_channels( @@ -91,9 +96,14 @@ def refresh_view(self) -> None: def _current_generators(self) -> Optional[Dict[ChannelName, Features]]: """The channels of the reconstruction in front of the panel, where one is.""" + envelopes = self._reconstruction_envelopes() + return None if envelopes is None else envelopes.channels + + def _reconstruction_envelopes(self) -> Optional[ChannelEnvelopesViewModel]: + """The reconstruction in front of the panel, where it holds one and no instrument.""" match self._editor.edited_instrument(): - case ReconstructionEdit() as edit: - return edit.channels + case ChannelEnvelopesViewModel() as envelopes: + return envelopes case _: return None diff --git a/src/sampletones_application/logic/reconstruction/manager.py b/src/sampletones_application/logic/reconstruction/manager.py index 453b92d49..2e4ccd7cd 100644 --- a/src/sampletones_application/logic/reconstruction/manager.py +++ b/src/sampletones_application/logic/reconstruction/manager.py @@ -5,15 +5,17 @@ from sampletones_application.layout.behavior.scheduling.scheduling import SchedulingBehavior from sampletones_application.logic.reconstruction.data import ReconstructionData -from sampletones_application.logic.reconstruction.feature import FeatureData +from sampletones_application.logic.reconstruction.envelopes import heard_envelopes from sampletones_application.logic.reconstruction.listening import StemListening from sampletones_application.logic.reconstruction.session import ReconstructionSession from sampletones_application.utils.callbacks.queue import CallbackQueue +from sampletones_application.view_model.reconstruction.envelopes import ( + ChannelEnvelopesViewModel, +) from sampletones_core.reconstructions import Reconstruction from sampletones_shared.logger import logger from sampletones_shared.types.callback import VoidCallback from sampletones_shared.utils.callbacks import CallbackMixin -from sampletones_shared.utils.hashing import hash_model from sampletones_shared.utils.system.paths import first_missing from sampletones_shared.utils.system.reveal.selection import open_paths_in_explorer @@ -33,10 +35,8 @@ def __init__(self, *, scheduling: SchedulingBehavior) -> None: self._scheduling = scheduling self._session: ReconstructionSession = ReconstructionSession() self._current_reconstruction: Optional[ReconstructionData] = None - self._current_features: Optional[FeatureData] = None + self._current_features: Optional[ChannelEnvelopesViewModel] = None self._listening: StemListening = StemListening() - self._reconstruction_hash: str = "" - self._coefficient: float = 1.0 self.on_reconstruction_loaded: Optional[VoidCallback] = None self.on_reconstruction_closed: Optional[VoidCallback] = None @@ -74,13 +74,12 @@ def load_reconstruction_object( def _adopt_reconstruction(self, reconstruction_data: ReconstructionData) -> None: """Makes ``reconstruction_data`` the open document and refreshes its derived state. - The coefficient, the reader's listening choice and the cached features track whichever - reconstruction is open, so every rebinding funnels through here to recompute them in one - place. The listening is carried onto the new record before the features are read, so the - envelopes answer for the part the reader is listening to as it now stands. + The reader's listening choice and the cached features track whichever reconstruction is + open, so every rebinding funnels through here to recompute them in one place. The + listening is carried onto the new record before the features are read, so the envelopes + answer for the part the reader is listening to as it now stands. """ self._current_reconstruction = reconstruction_data - self._coefficient = reconstruction_data.reconstruction.coefficient self._listening.adopt(reconstruction_data.reconstruction.stems_data) self._load_reconstruction_features() @@ -94,8 +93,7 @@ def _load_reconstruction_features(self) -> None: raise RuntimeError("No reconstruction is loaded when trying to load features") reconstruction = self._current_reconstruction.reconstruction - self._current_features = FeatureData.heard(reconstruction, self._listening.selection) - self._reconstruction_hash = hash_model(reconstruction) + self._current_features = heard_envelopes(reconstruction, self._listening.selection) def refresh_features(self) -> None: """Reads the envelopes again after a change to what the reader is listening to. @@ -184,8 +182,6 @@ def close_reconstruction(self) -> None: self._current_reconstruction = None self._current_features = None self._listening.release() - self._reconstruction_hash = "" - self._coefficient = 1.0 self._session.mark_closed() CallbackQueue.add( self.call, @@ -213,7 +209,7 @@ def current_reconstruction(self) -> Optional[ReconstructionData]: return self._current_reconstruction @property - def current_features(self) -> Optional[FeatureData]: + def current_features(self) -> Optional[ChannelEnvelopesViewModel]: return self._current_features @property diff --git a/src/sampletones_application/logic/reconstruction/ownership.py b/src/sampletones_application/logic/reconstruction/ownership.py new file mode 100644 index 000000000..5d20ddac3 --- /dev/null +++ b/src/sampletones_application/logic/reconstruction/ownership.py @@ -0,0 +1,105 @@ +from typing import AbstractSet, Callable, Dict, Final, Mapping, Sequence + +from sampletones_application.logic.reconstruction.listening import offered_channels +from sampletones_application.view_model.shared.ownership import ( + OwnershipLaneViewModel, + OwnershipRunViewModel, +) +from sampletones_core.constants.algorithm import RESTING_STEM_ID +from sampletones_core.constants.enums import ChannelName +from sampletones_core.reconstructions.reconstruction.stems.data import StemsData +from sampletones_core.reconstructions.reconstruction.stems.ownership import heard_frame, owner_runs + +DISTINGUISHABLE_OWNERS: Final[int] = 2 + +HeardOn = Callable[[ChannelName], AbstractSet[int]] + + +def record_positions(stems_data: StemsData) -> Dict[int, int]: + """Where each recording's entry stands on the record, which is what picks its color. + + Every surface painting a recording reads this one ordering, so a stretch under the waveform, + a stretch under an instrument's bars and the swatch beside a name are drawn in the same color. + + Args: + stems_data: The record the document carries. + + Returns: + Dict[int, int]: The place each recording's entry stands at, by stem id. + """ + return {entry.id: index for index, entry in enumerate(stems_data.config.entries)} + + +def tells_owners_apart(stems_data: StemsData) -> bool: + """Whether the document holds owners a color has something to tell apart. + + An owner is a recording the record names, or the row the frames a reader wrote gather under: + both take a color of their own and both stand on the stems card, so a document holding one + recording and an edit beside it has two owners to tell apart. + + Args: + stems_data: The record the document carries. + + Returns: + bool: True where two or more owners stand on the record. + """ + return len(offered_channels(stems_data)) >= DISTINGUISHABLE_OWNERS + + +def ownership_lanes( + stems_data: StemsData, + assignments: Mapping[ChannelName, Sequence[int]], + heard_on: HeardOn, +) -> Dict[ChannelName, OwnershipLaneViewModel]: + """A lane per channel of ``assignments``, each divided into the stretches its owners hold. + + Each surface states its own reading as the assignments it passes — which channels stand, and + how far each lane runs — so the rule dividing a channel into stretches is written once and + the ribbon under the waveform and the band under an instrument's bars agree by construction. + A document answering to a single owner has nothing to tell apart and offers no lane. + + Args: + stems_data: The record the document carries. + assignments: The stem holding each frame, per channel the surface draws. + heard_on: The recordings the reader hears on a channel. + + Returns: + Dict[ChannelName, OwnershipLaneViewModel]: One lane per channel given, in that order. + """ + if not tells_owners_apart(stems_data): + return {} + + positions = record_positions(stems_data) + return { + channel_name: _ownership_lane(channel_name, stem_ids, positions, heard_on(channel_name)) + for channel_name, stem_ids in assignments.items() + } + + +def _ownership_lane( + channel_name: ChannelName, + stem_ids: Sequence[int], + positions: Mapping[int, int], + heard: AbstractSet[int], +) -> OwnershipLaneViewModel: + """One channel's lane: the stretches it divides into, each under the recording holding it. + + A stretch keeps the recording holding it and says whether the reader hears it there, so + the lane names an owner wherever the record does and shows the reader's choice on top of it. + A resting stretch answers to no recording, so it takes no stretch of its own and leaves the + ground of whatever surface draws the lane showing through. + """ + return OwnershipLaneViewModel( + channel_name=channel_name, + runs=tuple( + OwnershipRunViewModel( + start_frame=run.start, + end_frame=run.end, + stem_id=run.stem_id, + position=positions.get(run.stem_id, 0), + heard=heard_frame(run.stem_id, heard), + ) + for run in owner_runs(stem_ids) + if run.stem_id != RESTING_STEM_ID + ), + ) diff --git a/src/sampletones_application/logic/reconstruction/reconstruction.py b/src/sampletones_application/logic/reconstruction/reconstruction.py index a94d48151..22816f129 100644 --- a/src/sampletones_application/logic/reconstruction/reconstruction.py +++ b/src/sampletones_application/logic/reconstruction/reconstruction.py @@ -17,9 +17,15 @@ from sampletones_application.constants.sources import SourceKind from sampletones_application.logic.export.instrument.source import ExportableInstrument from sampletones_application.logic.reconstruction.data import ReconstructionData -from sampletones_application.logic.reconstruction.feature import FeatureData from sampletones_application.logic.reconstruction.listening import StemListening from sampletones_application.logic.reconstruction.manager import ReconstructionManager +from sampletones_application.logic.reconstruction.ownership import ( + ownership_lanes, + record_positions, +) +from sampletones_application.view_model.reconstruction.envelopes import ( + ChannelEnvelopesViewModel, +) from sampletones_application.view_model.reconstruction.paths.path import ( ReconstructionPathViewModel, ) @@ -33,18 +39,14 @@ ReconstructionStemsViewModel, ) from sampletones_application.view_model.shared.audio_data import AudioData -from sampletones_application.view_model.shared.ownership import ( - OwnershipLaneViewModel, - OwnershipRibbonViewModel, - OwnershipRunViewModel, -) +from sampletones_application.view_model.shared.ownership import OwnershipRibbonViewModel from sampletones_application.view_model.shared.stems import ( StemRowViewModel, StemsListViewModel, ) from sampletones_application.view_model.shared.waveform_data import WaveformData from sampletones_core.configs.library import InstructionsLibraryConfig -from sampletones_core.constants.algorithm import AUTHORED_STEM_ID, RESTING_STEM_ID +from sampletones_core.constants.algorithm import AUTHORED_STEM_ID from sampletones_core.constants.enums import AudioSourceType, ChannelName from sampletones_core.exporters.feature import Features from sampletones_core.exporters.naming import instrument_slice_name @@ -57,7 +59,6 @@ ) from sampletones_core.exports.scope import ExportScope from sampletones_core.reconstructions.reconstruction.stems.data import StemsData -from sampletones_core.reconstructions.reconstruction.stems.ownership import heard_frame, owner_runs from sampletones_core.reconstructions.reconstruction.stems.selection import ( StemSelection, ) @@ -72,7 +73,6 @@ ) EMPTY_STEMS_LIST: Final[StemsListViewModel] = StemsListViewModel.empty() -DISTINGUISHABLE_RECORDINGS: Final[int] = 2 class ExportServiceProtocol(Protocol): @@ -317,62 +317,22 @@ def _build_ownership_ribbon( channel the reader has switched off keeps its lane and stands empty, since the lanes answer for the document while what fills them answers for the listening: the rows beneath the waveform hold still while a reader picks their way through it. A document answering to - one recording alone has nothing to tell apart, so it offers no lanes. + one owner alone has nothing to tell apart, so it offers no lanes. """ stems_data = reconstruction_data.reconstruction.stems_data - positions = self._record_positions(stems_data) - if len(positions) < DISTINGUISHABLE_RECORDINGS: - return OwnershipRibbonViewModel.empty() - owned = stems_data.assignments_by_channel - lanes = tuple( - self._ownership_lane(channel_name, owned[channel_name], positions) + assignments: Dict[ChannelName, Sequence[int]] = { + channel_name: owned[channel_name] if channel_name in self._selected_channels else () for channel_name in self._in_channel_order(self._playing_channels) if channel_name in owned - ) + } + lanes = tuple(ownership_lanes(stems_data, assignments, self.heard_on).values()) return OwnershipRibbonViewModel( lanes=lanes, frame_length=reconstruction_data.reconstruction.config.frame_length, total_frames=max((len(owned[lane.channel_name]) for lane in lanes), default=0), ) - @staticmethod - def _record_positions(stems_data: StemsData) -> Dict[int, int]: - """Where each recording's entry stands on the record, which is what picks its color. - - The ribbon and the row beside it read this one ordering, so a stretch and the name it - answers to are drawn in the same color. - """ - return {entry.id: index for index, entry in enumerate(stems_data.config.entries)} - - def _ownership_lane( - self, - channel_name: ChannelName, - stem_ids: Sequence[int], - positions: Dict[int, int], - ) -> OwnershipLaneViewModel: - """One channel's lane: the stretches it divides into, each under the recording heard on it. - - A channel the reader has switched off is not listened to at all, so its lane divides into - nothing and the row it stands in shows the ground it is laid on. - """ - if channel_name not in self._selected_channels: - return OwnershipLaneViewModel(channel_name=channel_name, runs=()) - - heard = self.heard_on(channel_name) - return OwnershipLaneViewModel( - channel_name=channel_name, - runs=tuple( - OwnershipRunViewModel( - start_frame=run.start, - end_frame=run.end, - stem_id=run.stem_id if heard_frame(run.stem_id, heard) else RESTING_STEM_ID, - position=positions.get(run.stem_id, 0), - ) - for run in owner_runs(stem_ids) - ), - ) - def heard_on(self, channel_name: ChannelName) -> FrozenSet[int]: """The recordings the reader hears on one channel, which is the scope an edit writes in. @@ -401,7 +361,7 @@ def _build_stems_view_model( ) entries = stems_data.config.entries_by_id - positions = self._record_positions(stems_data) + positions = record_positions(stems_data) levels = self._levels_with_edits(stems_data) rows = tuple( self._stem_row( @@ -514,7 +474,7 @@ def exportable_instrument( ), ) - def _heard_features(self) -> FeatureData: + def _heard_features(self) -> ChannelEnvelopesViewModel: """The envelopes of the part the reader is listening to, as the open document reads them. Raises: diff --git a/src/sampletones_application/logic/render/logic.py b/src/sampletones_application/logic/render/logic.py index 7d2a0ead8..a5a5baf1e 100644 --- a/src/sampletones_application/logic/render/logic.py +++ b/src/sampletones_application/logic/render/logic.py @@ -1,6 +1,7 @@ from pathlib import Path from typing import Callable, Dict, Optional, Tuple +from sampletones_application.categories.estimate import time_estimation from sampletones_application.categories.manager import LanguageManager from sampletones_application.config.managers.config import ConfigManager from sampletones_application.config.managers.session import SessionManager @@ -31,7 +32,6 @@ available_depths, ) from sampletones_core.constants.enums import ALL_CHANNELS -from sampletones_core.parallelization import ETAEstimator from sampletones_shared.constants.project import DEFAULT_EXPORT_NAME from sampletones_shared.logger import logger from sampletones_shared.types.callback import PathCallback, VoidCallback @@ -68,11 +68,11 @@ def __init__( self._session_manager = session_manager self._service = render_service self._is_operation_active = is_operation_active + self._language_manager = language_manager self._msg_canceling = language_manager["settings.render.message.status_canceling"] self._msg_canceled = language_manager["settings.render.message.status_canceled"] self._msg_completed = language_manager["settings.render.message.status_completed"] self._msg_failed = language_manager["settings.render.message.status_failed"] - self._eta_template = language_manager["global.dialog.template.time_estimation"] self._stage_messages: Dict[RenderStage, str] = { RenderStage.SYNTHESIS: language_manager["settings.render.message.status_synthesis"], RenderStage.ENCODING: language_manager["settings.render.message.status_encoding"], @@ -220,13 +220,8 @@ def _handle_progress(self, progress: ServiceProgress[RenderStage]) -> None: self._report(status_text, progress.fraction) def _stage_status(self, stage: RenderStage, eta_seconds: Optional[float]) -> str: - """What the pass is doing, and how long it has left where an estimate stands.""" - status_text = self._stage_messages[stage] - eta_string = ETAEstimator.format_duration(eta_seconds) - if eta_string: - status_text += self._eta_template.format(eta_string=eta_string) - - return status_text + """What the pass is doing, and how long it has left.""" + return self._stage_messages[stage] + time_estimation(self._language_manager, eta_seconds) def _on_render_complete(self, destination: Path) -> None: self._phase = RenderPhase.COMPLETED diff --git a/src/sampletones_application/tags/graphs.py b/src/sampletones_application/tags/graphs.py index c53ab9b06..701c518a9 100644 --- a/src/sampletones_application/tags/graphs.py +++ b/src/sampletones_application/tags/graphs.py @@ -33,7 +33,10 @@ SUF_WAVEFORM_POSITION_INDICATOR = "position_indicator" SUF_WAVEFORM_OVERLAY = "overlay" SUF_RIBBON_LANE = "lane" -SUF_RIBBON_RUN = "run" +SUF_RIBBON_GROUND = "ground" +SUF_OWNERSHIP_RUN = "ownership_run" +SUF_OWNERSHIP_HEARD = "heard" +SUF_OWNERSHIP_LEFT_OUT = "left_out" SUF_BAR_PLOT_ZERO_LINE = "zero_line" SUF_BAR_PLOT_HOVER_BAR = "hover_bar" SUF_HANDLER_MOUSE = compose_tag("handler", "mouse") diff --git a/src/sampletones_application/ui/elements/dialog.py b/src/sampletones_application/ui/elements/dialog.py index c7a6bd445..a265b584c 100644 --- a/src/sampletones_application/ui/elements/dialog.py +++ b/src/sampletones_application/ui/elements/dialog.py @@ -1,6 +1,7 @@ from abc import ABC from typing import Final, List, Optional +from sampletones_application.layout.primitives import DialogGeometry from sampletones_application.tags.general import TAG_GLOBAL_THEME_DIALOG from sampletones_application.ui.elements.window import GUIWindow from sampletones_application.ui.themes.registry import ThemeRegistry @@ -30,8 +31,7 @@ class GUIDialogWindow(GUIWindow, ABC): def __init__( self, tag: str, - width: int, - height: int, + geometry: DialogGeometry, *, key_router: KeyRouter, shortcut_source: ShortcutSource, @@ -40,11 +40,7 @@ def __init__( self._shortcuts = shortcut_source self._navigator: Optional[DialogKeyboardNavigator] = None - super().__init__( - tag, - width, - height, - ) + super().__init__(tag, geometry) def _install_navigation( self, diff --git a/src/sampletones_application/ui/elements/graphs/bar.py b/src/sampletones_application/ui/elements/graphs/bar.py index 4091f54c5..cd6ef5f14 100644 --- a/src/sampletones_application/ui/elements/graphs/bar.py +++ b/src/sampletones_application/ui/elements/graphs/bar.py @@ -208,6 +208,25 @@ def load_data( self._update_ticks() self._update_ranges() + def reserve_band(self, share: float) -> Tuple[float, float]: + """Keeps a band beneath the plotted values, and answers where that band lies. + + A stretch drawn under the bars stands in a band of its own, so the plot lowers what it + spans by that share and the bars keep every value they reach. The reserve is measured + against the range the plot was built with, so reserving twice keeps one band. + + Args: + share: How much of the built range the band takes, beneath it. + + Returns: + Tuple[float, float]: The band's lower and upper edge. + """ + low, high = self._default_y_range + height = (high - low) * share + self.y_range = (low - height, high) + self._update_axes_limits() + return low - height, low + def _set_layer( self, layer: BarLayer, @@ -337,9 +356,19 @@ def _on_mouse_action(self, _sender: Sender) -> None: self._set_hover_bar_position(bar_index, clamped_y) self.call(self.on_bar_point_hovered, name, bar_index) - if dpg.is_mouse_button_down(dpg.mvMouseButton_Left) or dpg.is_mouse_button_clicked(dpg.mvMouseButton_Left): + if self._presses_a_bar(mouse_y) and ( + dpg.is_mouse_button_down(dpg.mvMouseButton_Left) or dpg.is_mouse_button_clicked(dpg.mvMouseButton_Left) + ): self._draw_bar(layer, bar_index, clamped_y, previous_stroke) + def _presses_a_bar(self, mouse_y: float) -> bool: + """Whether a press at ``mouse_y`` stands on the grid the bars are drawn across. + + A band reserved beneath the bars lies inside the plot and reads which stretch belongs to + whom, so a press there answers what the band shows and the values above it stand. + """ + return mouse_y >= self._default_y_range[0] + def _draw_bar( self, layer: BarLayer, diff --git a/src/sampletones_application/ui/elements/graphs/ownership.py b/src/sampletones_application/ui/elements/graphs/ownership.py new file mode 100644 index 000000000..383e96e83 --- /dev/null +++ b/src/sampletones_application/ui/elements/graphs/ownership.py @@ -0,0 +1,88 @@ +from typing import Dict, List, Sequence, Tuple + +import dearpygui.dearpygui as dpg + +from sampletones_application.layout.general.colors.stem import StemColors +from sampletones_application.tags.compose import compose_tag +from sampletones_application.tags.graphs import ( + SUF_GRAPH_THEME, + SUF_OWNERSHIP_HEARD, + SUF_OWNERSHIP_LEFT_OUT, + SUF_OWNERSHIP_RUN, +) +from sampletones_application.utils.gui.dpg import dpg_bind_item_theme, dpg_delete_item +from sampletones_application.utils.gui.palette.dpg import dpg_add_palette_theme_color +from sampletones_application.view_model.shared.ownership import OwnershipRunViewModel + + +class OwnershipRuns: + """Paints the recordings behind a stretch of a plot, each run a flat bar in its own color. + + Two surfaces show a frame's owner — the ribbon under the waveform and the stretch under an + instrument's bars — so both paint through this, and a recording is one color wherever it is + shown. Each surface states the span its own axis counts in and the band the bars sit in, so + the same runs land under a waveform read in samples and under an envelope read in frames. + + A theme is named after the reading it paints — the recording, the place it stands on the + record and whether the reader hears it — so two stretches of one reading share a theme + however many times a document divides, and a name reads the same from one run of the + application to the next. + """ + + def __init__(self, stem_colors: StemColors) -> None: + self._stem_colors = stem_colors + self._painted: Dict[str, List[str]] = {} + + def paint( + self, + y_axis_tag: str, + runs: Sequence[OwnershipRunViewModel], + *, + frame_span: float, + band: Tuple[float, float], + ) -> None: + """Draws one lane's stretches into an axis. + + Args: + y_axis_tag: The axis the stretches are drawn on. + runs: The stretches, in frame order. + frame_span: What one frame measures on the axis the plot counts along. + band: The lower and upper edge the stretches are drawn between. + """ + self._clear(y_axis_tag) + bottom, top = band + painted: List[str] = [] + for run in runs: + series_tag = compose_tag(y_axis_tag, SUF_OWNERSHIP_RUN, str(run.start_frame)) + dpg.add_shade_series( + [run.start_frame * frame_span, run.end_frame * frame_span], + y1=[bottom, bottom], + y2=[top, top], + tag=series_tag, + parent=y_axis_tag, + ) + self._bind(series_tag, run) + painted.append(series_tag) + + self._painted[y_axis_tag] = painted + + def _clear(self, y_axis_tag: str) -> None: + """Takes the stretches this axis carries away, which is what a repaint begins with.""" + for series_tag in self._painted.pop(y_axis_tag, []): + dpg_delete_item(series_tag) + + def _bind(self, series_tag: str, run: OwnershipRunViewModel) -> None: + """Binds the run its recording's fill, through the theme every like reading shares.""" + theme_tag = compose_tag( + SUF_OWNERSHIP_RUN, + SUF_GRAPH_THEME, + str(run.stem_id), + str(run.position), + SUF_OWNERSHIP_HEARD if run.heard else SUF_OWNERSHIP_LEFT_OUT, + ) + if not dpg.does_item_exist(theme_tag): + color = self._stem_colors.for_stem(run.stem_id, run.position, heard=run.heard) + with dpg.theme(tag=theme_tag), dpg.theme_component(dpg.mvShadeSeries): + dpg_add_palette_theme_color(dpg.mvPlotCol_Fill, color, category=dpg.mvThemeCat_Plots) + + dpg_bind_item_theme(series_tag, theme_tag) diff --git a/src/sampletones_application/ui/elements/graphs/ribbon.py b/src/sampletones_application/ui/elements/graphs/ribbon.py index 6dcb8e097..a10534a81 100644 --- a/src/sampletones_application/ui/elements/graphs/ribbon.py +++ b/src/sampletones_application/ui/elements/graphs/ribbon.py @@ -5,14 +5,11 @@ from sampletones_application.layout.general.colors.stem import StemColors from sampletones_application.layout.graphs import GraphsLayout from sampletones_application.tags.compose import compose_tag -from sampletones_application.tags.graphs import SUF_GRAPH_THEME, SUF_RIBBON_RUN -from sampletones_application.utils.gui.dpg import dpg_bind_item_theme, dpg_delete_children +from sampletones_application.tags.graphs import SUF_GRAPH_THEME, SUF_RIBBON_GROUND +from sampletones_application.ui.elements.graphs.ownership import OwnershipRuns +from sampletones_application.utils.gui.dpg import dpg_bind_item_theme from sampletones_application.utils.gui.palette.dpg import dpg_add_palette_theme_color -from sampletones_application.utils.palette.colors.base import BaseColor -from sampletones_application.view_model.shared.ownership import ( - OwnershipLaneViewModel, - OwnershipRibbonViewModel, -) +from sampletones_application.view_model.shared.ownership import OwnershipRibbonViewModel from sampletones_core.constants.enums import ChannelName @@ -44,12 +41,12 @@ def __init__( self._layout = layout self._stem_colors = stem_colors self._view_model: OwnershipRibbonViewModel = OwnershipRibbonViewModel.empty() - self._run_themes: Dict[BaseColor, str] = {} + self._runs = OwnershipRuns(stem_colors) def bind_theme(self) -> None: """Lays each lane's ground, which is the color a resting stretch shows.""" for channel_name, plot_tag in self._plot_tags.items(): - theme_tag = compose_tag(plot_tag, SUF_GRAPH_THEME, SUF_RIBBON_RUN) + theme_tag = compose_tag(plot_tag, SUF_GRAPH_THEME, SUF_RIBBON_GROUND) with dpg.theme(tag=theme_tag), dpg.theme_component(dpg.mvPlot): dpg_add_palette_theme_color( dpg.mvPlotCol_PlotBg, @@ -63,15 +60,18 @@ def update_view(self, view_model: OwnershipRibbonViewModel) -> None: """Repaints the lanes for what the reader is listening to.""" self._view_model = view_model drawn = {lane.channel_name: lane for lane in view_model.lanes} + bottom = self._layout.ribbon.lane_gap / 2.0 for channel_name, y_axis_tag in self._y_axis_tags.items(): if not dpg.does_item_exist(y_axis_tag): continue - dpg_delete_children(y_axis_tag) lane = drawn.get(channel_name) - if lane is not None: - self._draw_lane(channel_name, lane, view_model) - + self._runs.paint( + y_axis_tag, + lane.runs if lane is not None else (), + frame_span=view_model.frame_length, + band=(bottom, 1.0 - bottom), + ) dpg.set_axis_limits(y_axis_tag, 0.0, 1.0) dpg.set_axis_limits_constraints(y_axis_tag, 0.0, 1.0) @@ -92,39 +92,3 @@ def lane_heights(self) -> Dict[ChannelName, int]: def lane_channels(self) -> List[ChannelName]: """The channels the lanes stand for, in the order they are drawn.""" return [lane.channel_name for lane in self._view_model.lanes] - - def _draw_lane( - self, - channel_name: ChannelName, - lane: OwnershipLaneViewModel, - view_model: OwnershipRibbonViewModel, - ) -> None: - """Paints one channel's stretches, each run a flat bar in its recording's color.""" - y_axis_tag = self._y_axis_tags[channel_name] - bottom = self._layout.ribbon.lane_gap / 2.0 - top = 1.0 - bottom - for run in lane.runs: - series_tag = compose_tag(y_axis_tag, SUF_RIBBON_RUN, str(run.start_frame)) - dpg.add_shade_series( - [ - run.start_frame * view_model.frame_length, - run.end_frame * view_model.frame_length, - ], - y1=[bottom, bottom], - y2=[top, top], - tag=series_tag, - parent=y_axis_tag, - ) - self._bind_run_theme(series_tag, self._stem_colors.for_stem(run.stem_id, run.position)) - - def _bind_run_theme(self, series_tag: str, color: BaseColor) -> None: - """Binds the run its recording's fill, reusing the theme two runs of one color share.""" - theme_tag = self._run_themes.get(color) - if theme_tag is None: - theme_tag = compose_tag(SUF_RIBBON_RUN, SUF_GRAPH_THEME, str(len(self._run_themes))) - with dpg.theme(tag=theme_tag), dpg.theme_component(dpg.mvShadeSeries): - dpg_add_palette_theme_color(dpg.mvPlotCol_Fill, color, category=dpg.mvThemeCat_Plots) - - self._run_themes[color] = theme_tag - - dpg_bind_item_theme(series_tag, theme_tag) diff --git a/src/sampletones_application/ui/elements/seeded.py b/src/sampletones_application/ui/elements/seeded.py new file mode 100644 index 000000000..2d6b44a1f --- /dev/null +++ b/src/sampletones_application/ui/elements/seeded.py @@ -0,0 +1,68 @@ +from abc import ABC, abstractmethod +from typing import Any, Generic, Optional, TypeVar + +from sampletones_application.layout.primitives import DialogGeometry +from sampletones_application.ui.elements.dialog import GUIDialogWindow +from sampletones_application.utils.gui.keyboard import KeyRouter +from sampletones_application.utils.gui.shortcuts.source import ShortcutSource + +ViewModel = TypeVar("ViewModel") + + +class GUISeededDialogWindow(GUIDialogWindow, ABC, Generic[ViewModel]): + """A dialog drawn from one view model, re-seeded for as long as it stands. + + What a settings form or a running job shows is settled before its tree is built, so seeding + and drawing are one act with a rebuild in between: :meth:`open` seeds the window and raises + it, :meth:`update_view` seeds it again and re-draws the controls in place, and the rebuild + itself has nothing left to capture. A window drawn before it was seeded reports that rather + than showing a form of empty fields. + """ + + def __init__( + self, + tag: str, + geometry: DialogGeometry, + *, + subject: str, + key_router: KeyRouter, + shortcut_source: ShortcutSource, + ) -> None: + self._subject = subject + self._view_model: Optional[ViewModel] = None + + super().__init__( + tag, + geometry, + key_router=key_router, + shortcut_source=shortcut_source, + ) + + def open(self, view_model: ViewModel) -> None: + """Shows the window seeded with what it is to draw.""" + self._view_model = view_model + self.show() + + def prepare(self, *_args: Any, **_kwargs: Any) -> None: + """The values drawn were seeded by :meth:`open` before the tree rebuilt.""" + + def update_view(self, view_model: ViewModel) -> None: + """Re-seeds the open window's controls from where its subject stands.""" + self._view_model = view_model + self._render() + + @property + def view_model(self) -> ViewModel: + """What the window draws. + + Raises: + SystemError: when the window is drawn before :meth:`open` seeds it. + """ + if self._view_model is None: + raise SystemError(f"The {self._subject} window is drawn only from a view model it was opened with") + + return self._view_model + + @abstractmethod + def _render(self) -> None: + """Draws the controls from the seeded view model.""" diff --git a/src/sampletones_application/ui/elements/trace.py b/src/sampletones_application/ui/elements/trace.py index b3d17a130..e3b2d60b8 100644 --- a/src/sampletones_application/ui/elements/trace.py +++ b/src/sampletones_application/ui/elements/trace.py @@ -30,6 +30,7 @@ def __init__( parent: str, exception: Exception, language_manager: LanguageManager, + height: int, theme: Optional[Theme] = None, button_theme: Optional[Theme] = None, ) -> None: @@ -43,6 +44,7 @@ def __init__( ), ) + self._height = height self._lbl_copy = language_manager["global.traceback.label.copy"] self.theme = ThemeRegistry.resolve(theme, TAG_GLOBAL_THEME_TRACEBACK) @@ -60,7 +62,7 @@ def __init__( default_value=self._text, multiline=True, readonly=True, - height=400, + height=self._height, width=-1, ) diff --git a/src/sampletones_application/ui/elements/window.py b/src/sampletones_application/ui/elements/window.py index 41a3138ab..a78009cdc 100644 --- a/src/sampletones_application/ui/elements/window.py +++ b/src/sampletones_application/ui/elements/window.py @@ -4,14 +4,15 @@ import dearpygui.dearpygui as dpg +from sampletones_application.layout.primitives import DialogGeometry from sampletones_application.tags.general import TAG_GLOBAL_THEME_DIALOG_WINDOW from sampletones_application.ui.elements.panel import GUIPanel from sampletones_application.ui.themes.registry import ThemeRegistry -from sampletones_application.utils.gui.align import center_when_settled +from sampletones_application.utils.gui.align import center_when_settled, viewport_center from sampletones_application.utils.gui.dpg import dpg_configure_item, dpg_delete_item from sampletones_application.utils.gui.frame import FrameCallbackManager +from sampletones_application.utils.placement import centered_position from sampletones_shared.types.callback import VoidCallback -from sampletones_shared.types.data import SerializedData class GUIWindow(GUIPanel, ABC): @@ -26,19 +27,21 @@ class GUIWindow(GUIPanel, ABC): A dialog that raises another modal — a prompt, a countdown — hands the screen over with ``yield_to`` and takes it back with ``resume``, which is what keeps - the two from competing for the one modal DearPyGui carries at a time. - - A window holding prose of a length it learns at the moment it opens sets - ``_fits_content``, which lets it grow past the height it states. + the two from competing for the one modal DearPyGui carries at a time. The + position a window was placed at is its own from then on, so the return brings + it back where it stood. A window claims the screen while it stands, so the reader answers it before going on. One reporting work already under way clears ``_claims_the_screen`` instead, which leaves the rest of the interface live beside it. """ - _fits_content: bool = False _claims_the_screen: bool = True + def __init__(self, tag: str, geometry: DialogGeometry) -> None: + self._geometry = geometry + super().__init__(tag, geometry.width, geometry.minimum_size[1]) + def yield_to(self, raise_modal: VoidCallback) -> None: """Steps off screen and runs ``raise_modal`` a frame later, so what it raises can open. @@ -68,47 +71,49 @@ def dialog_window( ) -> Iterator[None]: """Open this window's modal frame, with the block's widgets building inside it. - The window holds the width it states, which is what lets a field, a combo or a button - stretch across it: a stretched item measures one pixel inside the region it is offered, so - a window sized from its own content would take that pixel back on every frame. A stated - width settles the geometry in one pass and gives every dialog the same reading width - whatever it holds. - - A window that sets ``_fits_content`` reads its stated height as a floor and grows to hold - what it is given, so a prompt whose text wraps over several lines shows all of it. + The window opens at the size its geometry states and grows in height to hold more than + that, so a prompt whose text wraps over several lines and a form that unfolds a group + after opening both show the whole of what they hold. Its width is the one its geometry + states, held as the largest the window may take as well as the smallest, so an item + stretching across the window measures against a width that stands. A dialog offers the title bar's close button when ``on_close`` names what closing means, and omits it otherwise, so the only way out of a window is one the window answers for. """ - geometry: SerializedData = ( - {"min_size": (self.width, self.height), "autosize": True} if self._fits_content else {"height": self.height} - ) with dpg.window( tag=self.tag, label=label, - width=self.width, + width=self._geometry.width, + min_size=list(self._geometry.minimum_size), + max_size=list(self._geometry.maximum_size), + autosize=True, no_resize=True, no_collapse=True, no_close=on_close is None, on_close=on_close, modal=self._claims_the_screen, - **geometry, ): yield def show(self, *args: Any, **kwargs: Any) -> None: - """Builds this appearance's tree and centers it once the layout has measured it. - - A window's size is known to DearPyGui only after a frame has drawn it, so the center - waits for that frame to arrive on its own. Waiting for it in place would hold the render - thread, and a window is raised from wherever a result reaches the screen — including the - callback drain that runs between frames, where the frame being waited for is the one this - call is standing in the way of. + """Builds this appearance's tree, places it, and holds it centered as it takes its size. + + A window stating a height is placed before it is ever drawn, so the first frame carrying + it already shows it centered — which is what keeps it clear of the spot DearPyGui opens + an unplaced modal at. A window whose height its content settles has none to place from + and is centered on the frame that settles it. The drawn size is known only after a frame + has carried it, so the correction waits for those frames to arrive on their own: waiting + for one in place would hold the render thread, and a window is raised from wherever a + result reaches the screen — including the callback drain that runs between frames, where + the frame being waited for is the one this call stands in the way of. """ self.hide() self.prepare(*args, **kwargs) self.create_window() ThemeRegistry.get(TAG_GLOBAL_THEME_DIALOG_WINDOW).bind_to_item(self.tag) + if self._geometry.height is not None: + dpg.set_item_pos(self.tag, list(centered_position(viewport_center(), *self._geometry.minimum_size))) + center_when_settled(self.tag) def hide(self) -> None: diff --git a/src/sampletones_application/ui/panels/dialogs/audio_settings.py b/src/sampletones_application/ui/panels/dialogs/audio_settings.py index ae5c758c8..5617c34b8 100644 --- a/src/sampletones_application/ui/panels/dialogs/audio_settings.py +++ b/src/sampletones_application/ui/panels/dialogs/audio_settings.py @@ -68,8 +68,7 @@ def __init__( super().__init__( tag=TAG_SETTINGS_AUDIO_WINDOW, - width=layout.audio.window.width, - height=layout.audio.window.height, + geometry=layout.audio.window, key_router=key_router, shortcut_source=shortcut_source, ) diff --git a/src/sampletones_application/ui/panels/dialogs/countdown.py b/src/sampletones_application/ui/panels/dialogs/countdown.py index 593c25e6d..ace6c0a0c 100644 --- a/src/sampletones_application/ui/panels/dialogs/countdown.py +++ b/src/sampletones_application/ui/panels/dialogs/countdown.py @@ -2,7 +2,7 @@ import dearpygui.dearpygui as dpg -from sampletones_application.layout.primitives import Dimensions +from sampletones_application.layout.primitives import DialogGeometry from sampletones_application.tags.settings import ( TAG_SETTINGS_DISPLAY_BUTTON_KEEP, TAG_SETTINGS_DISPLAY_BUTTON_REVERT, @@ -37,7 +37,7 @@ class GUICountdownWindow(GUIDialogWindow): def __init__( self, *, - layout: Dimensions, + layout: DialogGeometry, title: str, message: str, remaining_format: str, @@ -58,8 +58,7 @@ def __init__( super().__init__( tag=TAG_SETTINGS_DISPLAY_WINDOW_COUNTDOWN, - width=layout.width, - height=layout.height, + geometry=layout, key_router=key_router, shortcut_source=shortcut_source, ) diff --git a/src/sampletones_application/ui/panels/dialogs/display_settings.py b/src/sampletones_application/ui/panels/dialogs/display_settings.py index e6aefb219..e828416bf 100644 --- a/src/sampletones_application/ui/panels/dialogs/display_settings.py +++ b/src/sampletones_application/ui/panels/dialogs/display_settings.py @@ -1,4 +1,4 @@ -from typing import Any, Callable, Optional +from typing import Callable, Optional import dearpygui.dearpygui as dpg @@ -16,8 +16,8 @@ TAG_SETTINGS_DISPLAY_WINDOW, ) from sampletones_application.ui.elements.button import GUIButton -from sampletones_application.ui.elements.dialog import GUIDialogWindow from sampletones_application.ui.elements.field import labeled_field, subheader +from sampletones_application.ui.elements.seeded import GUISeededDialogWindow from sampletones_application.utils.gui.align import table_wrapper from sampletones_application.utils.gui.dialog_navigation import FocusStop from sampletones_application.utils.gui.dpg import dpg_configure_item, dpg_set_value @@ -34,7 +34,7 @@ SettingsCallback = Callable[[DisplaySettings], None] -class GUIDisplaySettingsWindow(GUIDialogWindow): +class GUIDisplaySettingsWindow(GUISeededDialogWindow[DisplaySettingsViewModel]): """Modal form over how the application presents itself: its window, its pacing and its theme. Every control reports the whole edited state through ``on_settings_changed`` the moment it @@ -53,7 +53,6 @@ def __init__( ) -> None: self._language_manager = language_manager self._layout = layout - self._view_model: Optional[DisplaySettingsViewModel] = None self.on_settings_changed: Optional[SettingsCallback] = None self.on_commit: Optional[VoidCallback] = None @@ -62,26 +61,13 @@ def __init__( self._lbl_unlimited = language_manager["settings.display.label.unlimited_frame_rate"] super().__init__( + subject="display settings", tag=TAG_SETTINGS_DISPLAY_WINDOW, - width=layout.display.window.width, - height=layout.display.window.height, + geometry=layout.display.window, key_router=key_router, shortcut_source=shortcut_source, ) - def open(self, view_model: DisplaySettingsViewModel) -> None: - """Shows the window seeded with the given display settings.""" - self._view_model = view_model - self.show() - - def prepare(self, *_args: Any, **_kwargs: Any) -> None: - """The rendered values are seeded by :meth:`open` before the tree rebuilds.""" - - def update_view(self, view_model: DisplaySettingsViewModel) -> None: - """Re-seeds the controls of the open window from the given display settings.""" - self._view_model = view_model - self._render() - def create_window(self) -> None: with self.dialog_window( label=self._language_manager["settings.display.title.window_title"], @@ -117,7 +103,7 @@ def create_window(self) -> None: ) def _create_window_section(self) -> None: - view_model = self._require_view_model() + view_model = self.view_model subheader(self._language_manager["settings.display.title.section_window"]) with labeled_field( self._language_manager["settings.display.label.resolution"], @@ -145,7 +131,7 @@ def _create_window_section(self) -> None: ) def _create_pacing_section(self) -> None: - view_model = self._require_view_model() + view_model = self.view_model subheader(self._language_manager["settings.display.title.section_pacing"]) dpg.add_checkbox( tag=TAG_SETTINGS_DISPLAY_CHECKBOX_VSYNC, @@ -166,7 +152,7 @@ def _create_pacing_section(self) -> None: ) def _create_appearance_section(self) -> None: - view_model = self._require_view_model() + view_model = self.view_model subheader(self._language_manager["settings.display.title.section_appearance"]) with labeled_field( self._language_manager["settings.display.label.theme"], @@ -197,7 +183,7 @@ def _create_action_buttons(self) -> None: def _render(self) -> None: """Shows the standing selection, offering the size and frame controls while they apply.""" - view_model = self._require_view_model() + view_model = self.view_model dpg_configure_item( TAG_SETTINGS_DISPLAY_COMBO_RESOLUTION, items=list(view_model.resolution_items), @@ -223,50 +209,39 @@ def _render(self) -> None: dpg_set_value(TAG_SETTINGS_DISPLAY_COMBO_PALETTE, view_model.settings.palette) def _on_resolution_changed(self, _sender: Sender, app_data: str) -> None: - view_model = self._require_view_model() + view_model = self.view_model resolution = view_model.resolution_for_item(app_data) self._emit_window(view_model.settings.window.with_resolution(resolution)) def _on_borderless_changed(self, _sender: Sender, app_data: bool) -> None: - window = self._require_view_model().settings.window + window = self.view_model.settings.window self._emit_window(window.with_borderless(bool(app_data))) def _on_fullscreen_changed(self, _sender: Sender, app_data: bool) -> None: - window = self._require_view_model().settings.window + window = self.view_model.settings.window self._emit_window(window.with_fullscreen(bool(app_data))) def _on_vsync_changed(self, _sender: Sender, app_data: bool) -> None: - settings = self._require_view_model().settings + settings = self.view_model.settings self._emit(settings.with_vsync(bool(app_data))) def _on_frame_rate_changed(self, _sender: Sender, app_data: str) -> None: - view_model = self._require_view_model() + view_model = self.view_model frame_rate = view_model.frame_rate_for_item(app_data, self._lbl_unlimited) self._emit(view_model.settings.with_frame_rate(frame_rate)) def _on_palette_changed(self, _sender: Sender, app_data: str) -> None: - settings = self._require_view_model().settings + settings = self.view_model.settings self._emit(settings.with_palette(app_data)) def _emit(self, settings: DisplaySettings) -> None: self.call(self.on_settings_changed, settings) def _emit_window(self, window: WindowMode) -> None: - self._emit(self._require_view_model().settings.with_window(window)) + self._emit(self.view_model.settings.with_window(window)) def _request_commit(self) -> None: self.call(self.on_commit) def _request_cancel(self) -> None: self.call(self.on_cancel) - - def _require_view_model(self) -> DisplaySettingsViewModel: - """The settings on screen. - - Raises: - SystemError: when the window is drawn before :meth:`open` seeds it. - """ - if self._view_model is None: - raise SystemError("The display settings window is drawn from a view model it was opened with") - - return self._view_model diff --git a/src/sampletones_application/ui/panels/dialogs/export.py b/src/sampletones_application/ui/panels/dialogs/export.py index 5725fda24..8f0256782 100644 --- a/src/sampletones_application/ui/panels/dialogs/export.py +++ b/src/sampletones_application/ui/panels/dialogs/export.py @@ -1,4 +1,4 @@ -from typing import Any, Dict, Final, Optional +from typing import Dict, Final, Optional import dearpygui.dearpygui as dpg @@ -17,10 +17,10 @@ TAG_SETTINGS_EXPORT_WINDOW, ) from sampletones_application.ui.elements.button import GUIButton -from sampletones_application.ui.elements.dialog import GUIDialogWindow from sampletones_application.ui.elements.fonts.font import Font from sampletones_application.ui.elements.fonts.registry import FontRegistry from sampletones_application.ui.elements.layout.centered import centered +from sampletones_application.ui.elements.seeded import GUISeededDialogWindow from sampletones_application.utils.gui.dialog_navigation import FocusStop from sampletones_application.utils.gui.dpg import dpg_configure_item, dpg_set_value from sampletones_application.utils.gui.keyboard import KeyRouter @@ -34,7 +34,7 @@ RING_STYLE: Final[int] = 1 -class GUIExportWindow(GUIDialogWindow): +class GUIExportWindow(GUISeededDialogWindow[SongExportViewModel]): """Modal report over an export while it runs. The file itself was named in the system's own save dialog, so this window has one face: what @@ -45,8 +45,6 @@ class GUIExportWindow(GUIDialogWindow): Canceling is offered for as long as the run can still answer one. """ - _fits_content = True - def __init__( self, *, @@ -59,7 +57,6 @@ def __init__( self._language_manager = language_manager self._text_colors = text_colors self._indicator = layout.export.indicator - self._view_model: SongExportViewModel = SongExportViewModel.idle() self.on_cancel: Optional[VoidCallback] = None @@ -71,25 +68,12 @@ def __init__( super().__init__( tag=TAG_SETTINGS_EXPORT_WINDOW, - width=layout.export.window.width, - height=layout.export.window.height, + geometry=layout.export.window, + subject="export", key_router=key_router, shortcut_source=shortcut_source, ) - def open(self, view_model: SongExportViewModel) -> None: - """Shows the window over the run that has just begun.""" - self._view_model = view_model - self.show() - - def prepare(self, *_args: Any, **_kwargs: Any) -> None: - """The drawn values are seeded by :meth:`open` before the tree rebuilds.""" - - def update_view(self, view_model: SongExportViewModel) -> None: - """Re-draws the open window from where the run stands.""" - self._view_model = view_model - self._render() - def create_window(self) -> None: with self.dialog_window( label=self._language_manager["settings.export.title.window_title"], @@ -176,7 +160,7 @@ def _create_cancel(self) -> None: def _render(self) -> None: """Draws the run as it stands: what it has been through, and where the latest stage is.""" - view_model = self._view_model + view_model = self.view_model self._render_stages(view_model) dpg_configure_item( TAG_SETTINGS_EXPORT_GROUP_MEASURED, @@ -222,5 +206,5 @@ def _stage_tag(self, stage: ExportStage) -> str: return compose_tag(TAG_SETTINGS_EXPORT_TEXT_STAGE, stage.value) def _request_cancel(self) -> None: - if self._view_model.cancel_enabled: + if self.view_model.cancel_enabled: self.call(self.on_cancel) diff --git a/src/sampletones_application/ui/panels/dialogs/keybindings.py b/src/sampletones_application/ui/panels/dialogs/keybindings.py index b5ca658e5..9047f5310 100644 --- a/src/sampletones_application/ui/panels/dialogs/keybindings.py +++ b/src/sampletones_application/ui/panels/dialogs/keybindings.py @@ -1,4 +1,4 @@ -from typing import Any, Callable, Optional +from typing import Callable, Optional import dearpygui.dearpygui as dpg @@ -25,10 +25,10 @@ TAG_SETTINGS_KEYBINDINGS_WINDOW, ) from sampletones_application.ui.elements.button import GUIButton -from sampletones_application.ui.elements.dialog import GUIDialogWindow from sampletones_application.ui.elements.field import labeled_field from sampletones_application.ui.elements.fonts.font import Font from sampletones_application.ui.elements.fonts.registry import FontRegistry +from sampletones_application.ui.elements.seeded import GUISeededDialogWindow from sampletones_application.utils.gui.align import table_wrapper from sampletones_application.utils.gui.dialog_navigation import FocusStop from sampletones_application.utils.gui.dpg import dpg_configure_item, dpg_set_value @@ -47,7 +47,7 @@ CombinationCallback = Callable[[KeyCombination], None] -class GUIKeybindingsWindow(GUIDialogWindow): +class GUIKeybindingsWindow(GUISeededDialogWindow[KeybindingsViewModel]): """Modal form over the keys each action answers to, one row per action grouped by its scope. A row is given keys either way round: clicking its shortcut cell listens for the press to @@ -71,7 +71,6 @@ def __init__( self._language_manager = language_manager self._layout = layout self._capture: Optional[KeyCapture] = None - self._view_model: Optional[KeybindingsViewModel] = None self._filter = "" self.on_scheme_selected: Optional[StringCallback] = None @@ -88,25 +87,16 @@ def __init__( super().__init__( tag=TAG_SETTINGS_KEYBINDINGS_WINDOW, - width=layout.keybindings.window.width, - height=layout.keybindings.window.height, + geometry=layout.keybindings.window, + subject="keybindings", key_router=key_router, shortcut_source=shortcut_source, ) def open(self, view_model: KeybindingsViewModel) -> None: - """Shows the window listing the actions of the draft being edited.""" - self._view_model = view_model + """Shows the window listing the actions of the draft being edited, over every scope.""" self._filter = "" - self.show() - - def prepare(self, *_args: Any, **_kwargs: Any) -> None: - """The rendered values are seeded by :meth:`open` before the tree rebuilds.""" - - def update_view(self, view_model: KeybindingsViewModel) -> None: - """Re-reads the rows of the open window from the draft as it now stands.""" - self._view_model = view_model - self._render() + super().open(view_model) def create_window(self) -> None: with self.dialog_window( @@ -144,7 +134,7 @@ def create_window(self) -> None: ) def _create_scheme_field(self) -> None: - view_model = self._require_view_model() + view_model = self.view_model with labeled_field( self._label(KeybindingsElements.SCHEME), self._layout.label_width, @@ -189,7 +179,7 @@ def _create_action_list(self) -> None: init_width_or_weight=self._layout.keybindings.action_width, ) dpg.add_table_column(label=self._label(KeybindingsElements.SHORTCUT)) - for group in self._require_view_model().groups: + for group in self.view_model.groups: self._create_group(group) def _create_group(self, group: KeybindingGroup) -> None: @@ -274,7 +264,7 @@ def _teardown(self) -> None: def _render(self) -> None: """Shows each action's keys, the standing selection, and what the filter leaves listed.""" - view_model = self._require_view_model() + view_model = self.view_model dpg_set_value(TAG_SETTINGS_KEYBINDINGS_COMBO_SCHEME, view_model.scheme) dpg_set_value(TAG_SETTINGS_KEYBINDINGS_INPUT_SHORTCUT, view_model.combination) dpg_set_value(TAG_SETTINGS_KEYBINDINGS_TEXT_MESSAGE, view_model.message) @@ -409,14 +399,3 @@ def _require_capture(self) -> KeyCapture: raise SystemError("The keybindings window listens for a press only while it is open") return self._capture - - def _require_view_model(self) -> KeybindingsViewModel: - """The actions on screen. - - Raises: - SystemError: when the window is drawn before :meth:`open` seeds it. - """ - if self._view_model is None: - raise SystemError("The keybindings window is drawn from a view model it was opened with") - - return self._view_model diff --git a/src/sampletones_application/ui/panels/dialogs/nsf.py b/src/sampletones_application/ui/panels/dialogs/nsf.py index be6c633fe..1ea7a713d 100644 --- a/src/sampletones_application/ui/panels/dialogs/nsf.py +++ b/src/sampletones_application/ui/panels/dialogs/nsf.py @@ -1,5 +1,5 @@ from dataclasses import dataclass -from typing import Any, Callable, Dict, Final, Optional, Tuple +from typing import Callable, Dict, Final, Optional, Tuple import dearpygui.dearpygui as dpg @@ -34,11 +34,11 @@ TAG_SETTINGS_NSF_WINDOW, ) from sampletones_application.ui.elements.button import GUIButton -from sampletones_application.ui.elements.dialog import GUIDialogWindow from sampletones_application.ui.elements.field import labeled_field, subheader from sampletones_application.ui.elements.fonts.font import Font from sampletones_application.ui.elements.fonts.registry import FontRegistry from sampletones_application.ui.elements.path import GUIDestinationPathText +from sampletones_application.ui.elements.seeded import GUISeededDialogWindow from sampletones_application.ui.elements.status import GUIStatusBar from sampletones_application.ui.themes.channels import CHANNEL_THEME_TAGS from sampletones_application.ui.themes.registry import ThemeRegistry @@ -82,7 +82,7 @@ class HeaderField: edit: Callable[[NSFExportChoices, str], NSFExportChoices] -class GUINSFExportWindow(GUIDialogWindow): +class GUINSFExportWindow(GUISeededDialogWindow[NSFExportViewModel]): """Modal form over writing a project or a reconstruction as an NSF program. The dialog sets the export up and hands it over: the program's text, the channels it sounds, @@ -94,8 +94,6 @@ class GUINSFExportWindow(GUIDialogWindow): into keeps the text the reader types, and takes the text the header holds once it is left. """ - _fits_content = True - def __init__( self, *, @@ -112,7 +110,6 @@ def __init__( self._path_colors = path_colors self._text_colors = text_colors self._status_bar = status_bar - self._view_model: Optional[NSFExportViewModel] = None self._destination_text: Optional[GUIDestinationPathText] = None self._handler_tag = compose_tag(TAG_SETTINGS_NSF_WINDOW, SUF_HANDLER_REGISTRY) @@ -171,26 +168,13 @@ def __init__( } super().__init__( + subject="NSF export", tag=TAG_SETTINGS_NSF_WINDOW, - width=layout.nsf.window.width, - height=layout.nsf.window.height, + geometry=layout.nsf.window, key_router=key_router, shortcut_source=shortcut_source, ) - def open(self, view_model: NSFExportViewModel) -> None: - """Shows the window seeded with the export being set up.""" - self._view_model = view_model - self.show() - - def prepare(self, *_args: Any, **_kwargs: Any) -> None: - """The drawn values are seeded by :meth:`open` before the tree rebuilds.""" - - def update_view(self, view_model: NSFExportViewModel) -> None: - """Re-draws the open window from the choices the export stands at.""" - self._view_model = view_model - self._render() - def create_window(self) -> None: with self.dialog_window( label=self._language_manager["settings.nsf.title.window_title"], @@ -260,7 +244,7 @@ def _create_channels_section(self) -> None: @table_wrapper(columns=len(ChannelName.items()), height=0) def _create_channel_checkboxes(self) -> None: """Lays the channels out in one row, each tinted in its own color where the source sounds it.""" - view_model = self._require_view_model() + view_model = self.view_model for channel in ChannelName.items(): tag = self._channel_tag(channel) dpg.add_checkbox( @@ -349,7 +333,7 @@ def _create_destination(self) -> None: ) self._destination_text = GUIDestinationPathText( tag=TAG_SETTINGS_NSF_PATH_DESTINATION, - path=self._require_view_model().destination, + path=self.view_model.destination, parent=TAG_SETTINGS_NSF_GROUP_DESTINATION, color=self._path_colors.default, hover_color=self._path_colors.hover, @@ -383,7 +367,7 @@ def _create_field_handlers(self) -> None: def _render(self) -> None: """Draws the choices as they reconciled, around whatever field is being typed into.""" - view_model = self._require_view_model() + view_model = self.view_model self._render_fields(view_model) self._render_channels(view_model) self._render_repeat(view_model) @@ -438,11 +422,11 @@ def _on_text_changed(self, _sender: Sender, app_data: str, field: HeaderField) - self._emit(field.edit(self._choices(), app_data)) def _on_channel_changed(self, _sender: Sender, app_data: bool, channel: ChannelName) -> None: - view_model = self._require_view_model() + view_model = self.view_model self._emit(view_model.choices.with_channel(channel, bool(app_data), view_model.offer)) def _on_repeat_changed(self, _sender: Sender, app_data: str) -> None: - view_model = self._require_view_model() + view_model = self.view_model self._emit(view_model.choices.with_repeat(self._repeats_by_label[app_data], view_model.offer)) def _on_loop_frame_changed(self, _sender: Sender, app_data: str) -> None: @@ -452,11 +436,11 @@ def _on_loop_frame_changed(self, _sender: Sender, app_data: str) -> None: except ValueError: return - view_model = self._require_view_model() + view_model = self.view_model self._emit(view_model.choices.with_loop_frame(frame, view_model.offer)) def _on_scheme_changed(self, _sender: Sender, app_data: str) -> None: - view_model = self._require_view_model() + view_model = self.view_model self._emit(view_model.choices.with_scheme(self._schemes_by_label[app_data], view_model.offer)) def _emit(self, choices: NSFExportChoices) -> None: @@ -466,7 +450,7 @@ def _request_destination(self) -> None: self.call(self.on_browse) def _request_export(self) -> None: - if self._require_view_model().export_enabled: + if self.view_model.export_enabled: self.call(self.on_export) def _request_close(self) -> None: @@ -478,19 +462,8 @@ def _teardown(self) -> None: dpg_delete_item(self._handler_tag) def _choices(self) -> NSFExportChoices: - return self._require_view_model().choices + return self.view_model.choices @staticmethod def _channel_tag(channel: ChannelName) -> str: return compose_tag(TAG_SETTINGS_NSF_CHECKBOX_CHANNEL, channel.value) - - def _require_view_model(self) -> NSFExportViewModel: - """The export on screen. - - Raises: - SystemError: when the window is drawn before :meth:`open` seeds it. - """ - if self._view_model is None: - raise SystemError("The NSF export window is drawn from a view model it was opened with") - - return self._view_model diff --git a/src/sampletones_application/ui/panels/dialogs/project_properties.py b/src/sampletones_application/ui/panels/dialogs/project_properties.py index 1b8e9d188..7393a36b7 100644 --- a/src/sampletones_application/ui/panels/dialogs/project_properties.py +++ b/src/sampletones_application/ui/panels/dialogs/project_properties.py @@ -105,8 +105,7 @@ def __init__( super().__init__( tag=TAG_SETTINGS_PROPERTIES_WINDOW, - width=layout.window.width, - height=layout.window.height, + geometry=layout.window, key_router=key_router, shortcut_source=shortcut_source, ) diff --git a/src/sampletones_application/ui/panels/dialogs/render.py b/src/sampletones_application/ui/panels/dialogs/render.py index 459494d26..d1f91603f 100644 --- a/src/sampletones_application/ui/panels/dialogs/render.py +++ b/src/sampletones_application/ui/panels/dialogs/render.py @@ -1,4 +1,4 @@ -from typing import Any, Callable, Dict, Optional +from typing import Callable, Dict, Optional import dearpygui.dearpygui as dpg @@ -27,11 +27,11 @@ TAG_SETTINGS_RENDER_WINDOW, ) from sampletones_application.ui.elements.button import GUIButton -from sampletones_application.ui.elements.dialog import GUIDialogWindow from sampletones_application.ui.elements.field import labeled_field from sampletones_application.ui.elements.fonts.font import Font from sampletones_application.ui.elements.fonts.registry import FontRegistry from sampletones_application.ui.elements.path import GUIDestinationPathText +from sampletones_application.ui.elements.seeded import GUISeededDialogWindow from sampletones_application.ui.elements.status import GUIStatusBar from sampletones_application.utils.gui.align import table_wrapper from sampletones_application.utils.gui.dialog_navigation import FocusStop @@ -49,7 +49,7 @@ SettingsCallback = Callable[[SongRenderSettings], None] -class GUIRenderWindow(GUIDialogWindow): +class GUIRenderWindow(GUISeededDialogWindow[SongRenderViewModel]): """Modal form over writing the open song to an audio file. The dialog has two faces and shows one at a time: the setup, where the file is described and @@ -76,7 +76,6 @@ def __init__( self._layout = layout self._path_colors = path_colors self._status_bar = status_bar - self._view_model: Optional[SongRenderViewModel] = None self._destination_text: Optional[GUIDestinationPathText] = None self.on_settings_changed: Optional[SettingsCallback] = None @@ -106,26 +105,13 @@ def __init__( } super().__init__( + subject="render", tag=TAG_SETTINGS_RENDER_WINDOW, - width=layout.render.window.width, - height=layout.render.window.height, + geometry=layout.render.window, key_router=key_router, shortcut_source=shortcut_source, ) - def open(self, view_model: SongRenderViewModel) -> None: - """Shows the window seeded with the render being set up.""" - self._view_model = view_model - self.show() - - def prepare(self, *_args: Any, **_kwargs: Any) -> None: - """The rendered values are seeded by :meth:`open` before the tree rebuilds.""" - - def update_view(self, view_model: SongRenderViewModel) -> None: - """Re-seeds the controls of the open window from where the render stands.""" - self._view_model = view_model - self._render() - def create_window(self) -> None: with self.dialog_window( label=self._language_manager["settings.render.title.window_title"], @@ -259,7 +245,7 @@ def _create_destination(self) -> None: ) self._destination_text = GUIDestinationPathText( tag=TAG_SETTINGS_RENDER_PATH_DESTINATION, - path=self._require_view_model().destination, + path=self.view_model.destination, parent=TAG_SETTINGS_RENDER_GROUP_DESTINATION, color=self._path_colors.default, hover_color=self._path_colors.hover, @@ -303,7 +289,7 @@ def _create_progress(self) -> None: def _render(self) -> None: """Shows the face the phase calls for, with each choice standing at what it reconciled to.""" - view_model = self._require_view_model() + view_model = self.view_model self._render_setup(view_model) self._render_progress(view_model) @@ -429,22 +415,11 @@ def _request_close(self) -> None: A render already stopping, and one that has reported its outcome, answer neither — what they are waiting for is the service, which arrives on its own. """ - view_model = self._require_view_model() + view_model = self.view_model if view_model.cancel_enabled: self._request_cancel() elif view_model.setup_visible: self.call(self.on_close) def _settings(self) -> SongRenderSettings: - return self._require_view_model().settings - - def _require_view_model(self) -> SongRenderViewModel: - """The render on screen. - - Raises: - SystemError: when the window is drawn before :meth:`open` seeds it. - """ - if self._view_model is None: - raise SystemError("The render window is drawn from a view model it was opened with") - - return self._view_model + return self.view_model.settings diff --git a/src/sampletones_application/ui/panels/dialogs/scanning.py b/src/sampletones_application/ui/panels/dialogs/scanning.py index e0710f043..648465c16 100644 --- a/src/sampletones_application/ui/panels/dialogs/scanning.py +++ b/src/sampletones_application/ui/panels/dialogs/scanning.py @@ -57,8 +57,7 @@ def __init__( super().__init__( tag=TAG_MAIN_CONVERTER_WINDOW_SCAN, - width=layout.scan.width, - height=layout.scan.height, + geometry=layout.scan, ) def open(self, root: Path) -> None: diff --git a/src/sampletones_application/ui/panels/dialogs/stem_selection.py b/src/sampletones_application/ui/panels/dialogs/stem_selection.py index b210ceb50..ae255c469 100644 --- a/src/sampletones_application/ui/panels/dialogs/stem_selection.py +++ b/src/sampletones_application/ui/panels/dialogs/stem_selection.py @@ -100,8 +100,7 @@ def __init__( super().__init__( tag=TAG_MAIN_CONVERTER_WINDOW_STEM_SELECTION, - width=layout.stem_selection.width, - height=layout.stem_selection.height, + geometry=layout.stem_selection, key_router=key_router, shortcut_source=shortcut_source, ) diff --git a/src/sampletones_application/ui/panels/reconstruction/instruments/instruments.py b/src/sampletones_application/ui/panels/reconstruction/instruments/instruments.py index ebeeb7f7c..16a5705ca 100644 --- a/src/sampletones_application/ui/panels/reconstruction/instruments/instruments.py +++ b/src/sampletones_application/ui/panels/reconstruction/instruments/instruments.py @@ -1,5 +1,5 @@ from functools import partial -from typing import Any, Callable, Dict, List, Optional, Tuple +from typing import Any, Callable, Dict, Final, List, Optional, Tuple import dearpygui.dearpygui as dpg import numpy as np @@ -19,6 +19,7 @@ INSTRUMENT_CHANNEL, ) from sampletones_application.layout.general.colors.feature import FeatureColors +from sampletones_application.layout.general.colors.stem import StemColors from sampletones_application.layout.graphs import GraphsLayout from sampletones_application.tags.compose import compose_tag from sampletones_application.tags.general import ( @@ -53,6 +54,7 @@ from sampletones_application.ui.elements.fonts.font import Font from sampletones_application.ui.elements.fonts.registry import FontRegistry from sampletones_application.ui.elements.graphs.bar import GUIBarGraph +from sampletones_application.ui.elements.graphs.ownership import OwnershipRuns from sampletones_application.ui.elements.graphs.utils import extend_y_range from sampletones_application.ui.elements.layout.card import card from sampletones_application.ui.elements.layout.collapse import CollapseAxis @@ -82,10 +84,14 @@ from sampletones_application.utils.gui.keyboard.piano import PIANO_KEYS from sampletones_application.utils.gui.palette.dpg import dpg_set_palette_color from sampletones_application.utils.gui.tooltip import show_tooltip +from sampletones_application.view_model.reconstruction.envelopes import ( + ChannelEnvelopesViewModel, +) from sampletones_application.view_model.reconstruction.instruments import ( ReconstructionInstrumentsViewModel, ) from sampletones_application.view_model.shared.footprint import VoiceFootprintViewModel +from sampletones_application.view_model.shared.ownership import OwnershipLaneViewModel from sampletones_core.constants.enums import ( ChannelName, FeatureKey, @@ -114,6 +120,8 @@ from sampletones_shared.types.callback import VoidCallback from sampletones_shared.utils.arrays import clamp +ONE_SLOT_PER_FRAME: Final[float] = 1.0 + OnInstrumentExportCallback = Callable[[ChannelName], None] OnAuditionCallback = Callable[[int], None] OnReconstructionInstrumentHoveredCallback = Callable[[Optional[int]], None] @@ -131,6 +139,7 @@ def __init__( pitch_stepper_style: PitchStepperStyle, copy_width: int, feature_colors: FeatureColors, + stem_colors: StemColors, layout_graphs: GraphsLayout, language_manager: LanguageManager, status_bar: GUIStatusBar, @@ -161,6 +170,7 @@ def __init__( self._pitch_stepper_style = pitch_stepper_style self._copy_width = copy_width self._layout_graphs = layout_graphs + self._ownership = OwnershipRuns(stem_colors) self._feature_plot_configs = make_feature_plot_configs( feature_colors, language_manager, @@ -307,8 +317,12 @@ def _get_feature_text_group_tag( ) -> str: return compose_tag(self.tab_bar_tag, channel_name, feature_key, SUF_GRAPH_RAW_DATA) - def _get_feature_text_tag(self, text_group_tag: str) -> str: - return compose_tag(text_group_tag, SUF_TEXT) + def _get_feature_text_tag( + self, + channel_name: ChannelName, + feature_key: FeatureKey, + ) -> str: + return compose_tag(self._get_feature_text_group_tag(channel_name, feature_key), SUF_TEXT) def _get_feature_plot_tag( self, @@ -476,12 +490,10 @@ def _update_raw_data_text( feature_key: FeatureKey, envelope: Envelope[int], ) -> None: - text_group_tag = self._get_feature_text_group_tag( - channel_name, - feature_key, + dpg_set_value( + self._get_feature_text_tag(channel_name, feature_key), + format_envelope(envelope), ) - raw_data_tag = self._get_feature_text_tag(text_group_tag) - dpg_set_value(raw_data_tag, format_envelope(envelope)) self._show_sequence(channel_name, feature_key, envelope) def update_view( @@ -649,25 +661,27 @@ def _format_size(self, byte_count: int) -> str: def update_feature_data( self, - generators: Optional[Dict[ChannelName, Features]], + envelopes: Optional[ChannelEnvelopesViewModel], ) -> None: - if generators is None: + if envelopes is None: return for channel_name in ChannelName.items(): - generator_features = generators.get(channel_name) + generator_features = envelopes.channels.get(channel_name) if generator_features is None: continue self._update_generator_feature_data( channel_name, generator_features, + envelopes.lane(channel_name), ) def _update_generator_feature_data( self, channel_name: ChannelName, generator_features: Features, + lane: OwnershipLaneViewModel, ) -> None: initial_pitch = generator_features.initial_pitch self._apply_pitch_display(channel_name, initial_pitch) @@ -677,6 +691,7 @@ def _update_generator_feature_data( channel_name, generator_features, feature_key, + lane, ) def _update_generator_feature_display( @@ -684,10 +699,39 @@ def _update_generator_feature_display( channel_name: ChannelName, generator_features: Features, feature_key: FeatureKey, + lane: OwnershipLaneViewModel, ) -> None: envelope = self._feature_envelope(generator_features, feature_key) - self._update_generator_plot(channel_name, feature_key, _plotted_items(envelope)) + items = _plotted_items(envelope) + self._update_generator_plot(channel_name, feature_key, items) self._update_raw_data_text(channel_name, feature_key, envelope) + self._paint_ownership(channel_name, feature_key, lane, len(items)) + + def _paint_ownership( + self, + channel_name: ChannelName, + feature_key: FeatureKey, + lane: OwnershipLaneViewModel, + frame_count: int, + ) -> None: + """Paints the recording behind each frame in a band beneath that dimension's bars. + + The band stands under the frames the bars draw, so the two read column for column + whatever the dimension's own values reach. A dimension writing nothing, and a document + answering to one recording, have nothing to tell apart and give the band to the bars. + """ + plot = self.channel_plots.get(channel_name, {}).get(feature_key) + if plot is None: + return + + runs = lane.up_to(frame_count) + share = self._layout_graphs.bar_plot.ownership_band if runs else 0.0 + self._ownership.paint( + plot.y_axis_tag, + runs, + frame_span=ONE_SLOT_PER_FRAME, + band=plot.reserve_band(share), + ) def _feature_envelope( self, @@ -852,7 +896,6 @@ def _configure_plot_data( channel_name, feature_key, data, - plot.plot_tag, ), on_bar_point_hovered=self._on_bar_point_hovered, ) @@ -862,12 +905,9 @@ def _on_bar_point_clicked( channel_name: ChannelName, feature_key: FeatureKey, data: np.ndarray, - plot_tag: str, ) -> None: envelope = self._standing_sequence(channel_name, feature_key).with_items(tuple(int(value) for value in data)) - raw_data_tag = compose_tag(plot_tag, SUF_GRAPH_RAW_DATA) - dpg_set_value(raw_data_tag, format_envelope(envelope)) - self._show_sequence(channel_name, feature_key, envelope) + self._update_raw_data_text(channel_name, feature_key, envelope) self.call(self.on_envelope_changed, channel_name, feature_key, envelope) def _on_bar_point_hovered( @@ -897,7 +937,7 @@ def _add_raw_data_text( feature_key, ) raw_data_text = format_envelope(envelope) - raw_data_tag = self._get_feature_text_tag(text_group_tag) + raw_data_tag = self._get_feature_text_tag(channel_name, feature_key) copy_button_tag = compose_tag(text_group_tag, SUF_BUTTON_COPY) with dpg.group(tag=text_group_tag, parent=parent, horizontal=True): @@ -1000,8 +1040,7 @@ def _show_sequence( dimensions reach that file whole is visible before an export. """ self._sequences[(channel_name, feature_key)] = envelope - text_group_tag = self._get_feature_text_group_tag(channel_name, feature_key) - raw_data_tag = self._get_feature_text_tag(text_group_tag) + raw_data_tag = self._get_feature_text_tag(channel_name, feature_key) theme = self.warning_input_theme if is_shortened(feature_key, envelope) else self.theme theme.bind_to_item(raw_data_tag) diff --git a/src/sampletones_application/utils/gui/align.py b/src/sampletones_application/utils/gui/align.py index 5079b7376..7ae19830d 100644 --- a/src/sampletones_application/utils/gui/align.py +++ b/src/sampletones_application/utils/gui/align.py @@ -1,16 +1,19 @@ -from typing import Any, Callable, Final, Tuple +from functools import partial +from typing import Any, Callable, Optional, Tuple import dearpygui.dearpygui as dpg from sampletones_application.utils.gui.frame import FrameCallbackManager +from sampletones_application.utils.placement import centered_position -_SETTLE_FRAMES: Final[int] = 2 +def viewport_center() -> Tuple[int, int]: + """The middle of the space a window's position is measured in. -def get_center(width: int, height: int) -> Tuple[int, int]: - x = (dpg.get_viewport_width() - width) / 2 - y = (dpg.get_viewport_height() - height) / 2 - return round(x), round(y) + A position given to DearPyGui stands in the viewport's client area, so the center it is + measured against is read from the same space. + """ + return dpg.get_viewport_client_width() // 2, dpg.get_viewport_client_height() // 2 def center_item(tag: str) -> None: @@ -18,22 +21,32 @@ def center_item(tag: str) -> None: return width, height = dpg.get_item_rect_size(tag) - x, y = get_center(width, height) - dpg.set_item_pos(tag, [x, y]) + dpg.set_item_pos(tag, list(centered_position(viewport_center(), width, height))) def center_when_settled(tag: str) -> None: - """Centers an autosizing window once its content has reached its final size. + """Holds a window centered over the frames it takes its size in, and lets go once it settles. - An autosize window measures its content across the first couple of frames, so a stretch - table or wrapped text reaches its final width and height only on the second layout pass. - Deferring the center until then reads the settled size, so the window rests centered on its - first appearance the same way a reopened one does from its remembered size. + A window that takes the size its content asks for reaches it over the frames it is drawn + in: a form measures its fields on the second, and the keybindings list widens over a dozen + more. Centering the window against every size it is read at keeps it centered the whole way + there, and two readings that agree end the pass: a dialog can be dragged, and one that kept + measuring would drag it back. """ - FrameCallbackManager.set_frame_callback( - lambda: center_item(tag), - frame_count=_SETTLE_FRAMES, - ) + FrameCallbackManager.set_frame_callback(partial(_center_once_settled, tag, None)) + + +def _center_once_settled(tag: str, previous: Optional[Tuple[int, int]]) -> None: + """Centers the window where it now stands, and reads again while it is still growing.""" + if not dpg.does_item_exist(tag): + return + + width, height = dpg.get_item_rect_size(tag) + center_item(tag) + if previous == (width, height): + return + + FrameCallbackManager.set_frame_callback(partial(_center_once_settled, tag, (width, height))) def table_wrapper( diff --git a/src/sampletones_application/utils/gui/dialogs/renderer.py b/src/sampletones_application/utils/gui/dialogs/renderer.py index 84daefe61..51ffb52be 100644 --- a/src/sampletones_application/utils/gui/dialogs/renderer.py +++ b/src/sampletones_application/utils/gui/dialogs/renderer.py @@ -1,43 +1,36 @@ import re import uuid from pathlib import Path -from typing import Callable, Dict, List, Optional, Pattern, Tuple +from typing import Callable, Dict, Final, Optional, Pattern, Tuple import dearpygui.dearpygui as dpg from sampletones_application.categories.manager import LanguageManager from sampletones_application.layout.general import GeneralLayout +from sampletones_application.layout.primitives import DialogGeometry from sampletones_application.tags.compose import compose_tag from sampletones_application.tags.general import ( - SUF_BUTTON_OK, SUF_DIALOG_INFO, SUF_GROUP, SUF_PATH, TAG_GLOBAL_DIALOG_ERROR, TAG_GLOBAL_DIALOG_FILE_NOT_FOUND, TAG_GLOBAL_DIALOG_PATH_MESSAGE, - TAG_GLOBAL_THEME_DIALOG_WINDOW, ) from sampletones_application.tags.reconstructions import ( TAG_RECONSTRUCTIONS_RECONSTRUCTION_DIALOG_NOT_LOADED, ) -from sampletones_application.ui.elements.button import GUIButton from sampletones_application.ui.elements.fonts.font import Font from sampletones_application.ui.elements.fonts.registry import FontRegistry from sampletones_application.ui.elements.path import GUIPathText from sampletones_application.ui.elements.status import GUIStatusBar -from sampletones_application.ui.themes.registry import ThemeRegistry -from sampletones_application.utils.gui.align import center_when_settled -from sampletones_application.utils.gui.dialog_navigation import ( - DialogKeyboardNavigator, - FocusStop, -) from sampletones_application.utils.gui.dialogs.windows.confirmation import ( GUIConfirmationWindow, ) from sampletones_application.utils.gui.dialogs.windows.error import ( GUIErrorDialogWindow, ) +from sampletones_application.utils.gui.dialogs.windows.notice import show_notice from sampletones_application.utils.gui.dialogs.windows.save_confirmation import ( GUISaveConfirmationWindow, ) @@ -45,9 +38,11 @@ from sampletones_application.utils.gui.keyboard import KeyRouter from sampletones_application.utils.gui.palette.dpg import dpg_set_palette_color from sampletones_application.utils.gui.shortcuts.source import ShortcutSource -from sampletones_shared.types.callback import Callback, StringCallback, VoidCallback +from sampletones_shared.types.callback import Callback, StringCallback _TEMPLATE_PLACEHOLDER: Pattern[str] = re.compile(r"\{(\w+)\}") +DIALOG_TEXT_MARGIN: Final[int] = 10 +NOTICE_CLAIMS_THE_SCREEN: Final[bool] = True def get_dialog_tag(base_tag: str) -> str: @@ -55,85 +50,6 @@ def get_dialog_tag(base_tag: str) -> str: return compose_tag(base_tag, dialog_hash) -def _bind_dialog_theme(tag: str) -> None: - ThemeRegistry.get(TAG_GLOBAL_THEME_DIALOG_WINDOW).bind_to_item(tag) - - -def _install_navigation( - *, - window_tag: str, - stops: List[FocusStop], - on_escape: VoidCallback, - key_router: KeyRouter, - shortcut_source: ShortcutSource, - initial_index: int = 0, -) -> DialogKeyboardNavigator: - """Builds and installs the keyboard navigator that claims the keyboard for ``window_tag``.""" - navigator = DialogKeyboardNavigator( - window_tag=window_tag, - stops=stops, - on_escape=on_escape, - key_router=key_router, - shortcut_source=shortcut_source, - initial_index=initial_index, - ) - navigator.install() - return navigator - - -def _show_modal_dialog( - tag: str, - title: str, - content: StringCallback, - *, - ok_label: str, - width: int, - height: int, - key_router: KeyRouter, - shortcut_source: ShortcutSource, - modal: bool = True, -) -> None: - ok_button_tag = compose_tag(tag, SUF_BUTTON_OK) - navigator: Optional[DialogKeyboardNavigator] = None - - def close() -> None: - if navigator is not None: - navigator.dispose() - - dpg_delete_item(tag) - - with dpg.window( - label=title, - tag=tag, - modal=modal, - width=width, - min_size=(width, height), - no_resize=True, - autosize=True, - on_close=close, - ): - _bind_dialog_theme(tag) - content(tag) - dpg.add_separator() - GUIButton( - tag=ok_button_tag, - label=ok_label, - callback=close, - width=-1, - ) - - center_when_settled(tag) - - if modal: - navigator = _install_navigation( - window_tag=tag, - stops=[FocusStop.button(ok_button_tag, close)], - on_escape=close, - key_router=key_router, - shortcut_source=shortcut_source, - ) - - class DialogsRenderer: def __init__( self, @@ -148,20 +64,18 @@ def __init__( self._status_bar = status_bar self._router = key_router self._shortcuts = shortcut_source - self._default_width = layout.dialogs.default.width - self._default_height = layout.dialogs.default.height - self._error_width = layout.dialogs.error.width - self._error_height = layout.dialogs.error.height - self._confirmation_height = layout.dialogs.confirmation.height - self._default_wrap = layout.dialogs.default.width - 10 - self._error_wrap = layout.dialogs.error.width - 10 + self._default = layout.dialogs.default + self._error = layout.dialogs.error + self._confirmation = layout.dialogs.confirmation + self._traceback_height = layout.dialogs.traceback_height + self._default_wrap = layout.dialogs.default.width - DIALOG_TEXT_MARGIN + self._error_wrap = layout.dialogs.error.width - DIALOG_TEXT_MARGIN self._col_text_error = layout.colors.text.error self._col_text_highlight = layout.colors.text.highlight self._col_path = layout.colors.paths.default self._col_path_hover = layout.colors.paths.hover - self._recovery_width = layout.dialogs.recovery.width - self._recovery_height = layout.dialogs.recovery.height - self._recovery_wrap = layout.dialogs.recovery.width - 10 + self._recovery = layout.dialogs.recovery + self._recovery_wrap = layout.dialogs.recovery.width - DIALOG_TEXT_MARGIN self._msg_path = language_manager["global.status.message.path"] self._lbl_ok = language_manager["global.dialog.label.ok"] @@ -174,20 +88,30 @@ def show_modal( title: str, content: StringCallback, *, - width: Optional[int] = None, - height: Optional[int] = None, - modal: bool = True, + geometry: Optional[DialogGeometry] = None, ) -> None: - _show_modal_dialog( + """Raises a notice at ``geometry``, or at the size every dialog opens at by default.""" + self._notice(tag, title, content, geometry if geometry is not None else self._default) + + def _notice( + self, + tag: str, + title: str, + content: StringCallback, + geometry: DialogGeometry, + *, + claims_the_screen: bool = NOTICE_CLAIMS_THE_SCREEN, + ) -> None: + """Raises one notice the reader acknowledges, drawn at the size ``geometry`` states.""" + show_notice( tag, title, content, + geometry=geometry, ok_label=self._lbl_ok, key_router=self._router, shortcut_source=self._shortcuts, - width=width if width is not None else self._default_width, - height=height if height is not None else self._default_height, - modal=modal, + claims_the_screen=claims_the_screen, ) def show_info( @@ -203,17 +127,7 @@ def content(parent: str) -> None: info_tag = compose_tag(tag, SUF_DIALOG_INFO) dpg_delete_item(info_tag) - _show_modal_dialog( - tag=info_tag, - title=title, - content=content, - ok_label=self._lbl_ok, - key_router=self._router, - shortcut_source=self._shortcuts, - width=self._default_width, - height=self._default_height, - modal=modal, - ) + self._notice(info_tag, title, content, self._default, claims_the_screen=modal) def show_config_recovery( self, @@ -273,16 +187,12 @@ def content(parent: str) -> None: ) dpg_delete_item(tag) - _show_modal_dialog( - tag=tag, - title=self._language_manager["global.dialog.title.configuration_recovery"], - content=content, - ok_label=self._lbl_ok, - key_router=self._router, - shortcut_source=self._shortcuts, - width=self._recovery_width, - height=self._recovery_height, - modal=False, + self._notice( + tag, + self._language_manager["global.dialog.title.configuration_recovery"], + content, + self._recovery, + claims_the_screen=False, ) def _render_template_bold( @@ -330,9 +240,9 @@ def show_error( ) -> None: GUIErrorDialogWindow( tag=get_dialog_tag(TAG_GLOBAL_DIALOG_ERROR), - width=self._error_width, - height=self._error_height, + geometry=self._error, wrap=self._error_wrap, + traceback_height=self._traceback_height, language_manager=self._language_manager, error_color=self._col_text_error, key_router=self._router, @@ -351,16 +261,7 @@ def content(parent: str) -> None: ) dpg_set_palette_color(path_text, self._col_path) - _show_modal_dialog( - tag=tag, - title=self._language_manager["global.dialog.title.file_not_found"], - content=content, - ok_label=self._lbl_ok, - key_router=self._router, - shortcut_source=self._shortcuts, - width=self._error_width, - height=self._default_height, - ) + self._notice(tag, self._language_manager["global.dialog.title.file_not_found"], content, self._error) def show_confirmation( self, @@ -387,8 +288,7 @@ def show_confirmation( """ GUIConfirmationWindow( tag=get_dialog_tag(tag), - width=self._default_width, - height=self._confirmation_height, + geometry=self._confirmation, wrap=self._default_wrap, path_color=self._col_path, path_hover_color=self._col_path_hover, @@ -427,8 +327,7 @@ def show_save_confirmation( """ GUISaveConfirmationWindow( tag=get_dialog_tag(tag), - width=self._default_width, - height=self._confirmation_height, + geometry=self._confirmation, wrap=self._default_wrap, save_label=self._lbl_save, cancel_label=self._lbl_cancel, @@ -452,16 +351,12 @@ def content(parent: str) -> None: wrap=self._error_wrap, ) - _show_modal_dialog( - tag=tag, - title=self._language_manager["reconstructions.instruments.title.not_loaded_dialog"], - content=content, - ok_label=self._lbl_ok, - key_router=self._router, - shortcut_source=self._shortcuts, - width=self._error_width, - height=self._default_height, - modal=False, + self._notice( + tag, + self._language_manager["reconstructions.instruments.title.not_loaded_dialog"], + content, + self._error, + claims_the_screen=False, ) def show_message_with_path( @@ -486,14 +381,4 @@ def content(parent: str) -> None: status_bar=self._status_bar, ) - _show_modal_dialog( - tag=tag, - title=title, - content=content, - ok_label=self._lbl_ok, - key_router=self._router, - shortcut_source=self._shortcuts, - width=self._error_width, - height=self._default_height, - modal=False, - ) + self._notice(tag, title, content, self._error, claims_the_screen=False) diff --git a/src/sampletones_application/utils/gui/dialogs/windows/confirmation.py b/src/sampletones_application/utils/gui/dialogs/windows/confirmation.py index cbdb37d6b..ac15ca7b3 100644 --- a/src/sampletones_application/utils/gui/dialogs/windows/confirmation.py +++ b/src/sampletones_application/utils/gui/dialogs/windows/confirmation.py @@ -3,6 +3,7 @@ import dearpygui.dearpygui as dpg +from sampletones_application.layout.primitives import DialogGeometry from sampletones_application.tags.compose import compose_tag from sampletones_application.tags.general import ( SUF_BUTTON_CANCEL, @@ -35,14 +36,11 @@ class GUIConfirmationWindow(GUIDialogWindow): keyboard alone. """ - _fits_content = True - def __init__( self, tag: str, *, - width: int, - height: int, + geometry: DialogGeometry, wrap: int, path_color: BaseColor, path_hover_color: BaseColor, @@ -69,8 +67,7 @@ def __init__( super().__init__( tag, - width, - height, + geometry, key_router=key_router, shortcut_source=shortcut_source, ) diff --git a/src/sampletones_application/utils/gui/dialogs/windows/error.py b/src/sampletones_application/utils/gui/dialogs/windows/error.py index 3e7d63fc0..60731a1c6 100644 --- a/src/sampletones_application/utils/gui/dialogs/windows/error.py +++ b/src/sampletones_application/utils/gui/dialogs/windows/error.py @@ -3,6 +3,7 @@ import dearpygui.dearpygui as dpg from sampletones_application.categories.manager import LanguageManager +from sampletones_application.layout.primitives import DialogGeometry from sampletones_application.tags.compose import compose_tag from sampletones_application.tags.general import ( SUF_BUTTON_OK, @@ -12,7 +13,7 @@ from sampletones_application.ui.elements.button import GUIButton from sampletones_application.ui.elements.dialog import GUIDialogWindow from sampletones_application.ui.elements.trace import GUITraceback -from sampletones_application.utils.gui.align import table_wrapper +from sampletones_application.utils.gui.align import center_when_settled, table_wrapper from sampletones_application.utils.gui.dialog_navigation import FocusStop from sampletones_application.utils.gui.dpg import dpg_configure_item from sampletones_application.utils.gui.keyboard import KeyRouter @@ -29,15 +30,19 @@ class GUIErrorDialogWindow(GUIDialogWindow): The exception's name and text are drawn in the error color, the traceback starts hidden behind its toggle, and OK — the initially focused button — dismisses the prompt. The title-bar close reads the same way. + + Unfolding the traceback is the one gesture that changes a dialog's size while it stands, so + it centers the window again on the height it reaches: a report grown from its own height to + the traceback's would otherwise carry its buttons below the screen. """ def __init__( self, tag: str, *, - width: int, - height: int, + geometry: DialogGeometry, wrap: int, + traceback_height: int, language_manager: LanguageManager, error_color: BaseColor, key_router: KeyRouter, @@ -45,19 +50,19 @@ def __init__( ) -> None: self._language_manager = language_manager self._wrap = wrap + self._traceback_height = traceback_height self._error_color = error_color self._exception: Exception self._message: Optional[str] super().__init__( tag, - width, - height, + geometry, key_router=key_router, shortcut_source=shortcut_source, ) - def prepare(self, exception: Exception, message: Optional[str]) -> None: + def prepare(self, exception: Exception, message: Optional[str]) -> None: # pylint: disable=arguments-differ """Captures the failure the next appearance reports.""" self._exception = exception self._message = message @@ -67,13 +72,8 @@ def create_window(self) -> None: ok_button_tag = compose_tag(self.tag, SUF_BUTTON_OK) traceback: GUITraceback - with dpg.window( - tag=self.tag, + with self.dialog_window( label=self._language_manager["global.dialog.title.error"], - modal=True, - min_size=(self.width, self.height), - autosize=True, - no_scrollbar=False, on_close=self.hide, ): if self._message is not None: @@ -97,6 +97,7 @@ def create_window(self) -> None: parent=self.tag, exception=self._exception, language_manager=self._language_manager, + height=self._traceback_height, ) def toggle_traceback() -> None: @@ -109,6 +110,7 @@ def toggle_traceback() -> None: else self._language_manager["global.traceback.label.hide"] ), ) + center_when_settled(self.tag) dpg.add_separator() diff --git a/src/sampletones_application/utils/gui/dialogs/windows/notice.py b/src/sampletones_application/utils/gui/dialogs/windows/notice.py new file mode 100644 index 000000000..e80131ea5 --- /dev/null +++ b/src/sampletones_application/utils/gui/dialogs/windows/notice.py @@ -0,0 +1,94 @@ +from typing import Any + +import dearpygui.dearpygui as dpg + +from sampletones_application.layout.primitives import DialogGeometry +from sampletones_application.tags.compose import compose_tag +from sampletones_application.tags.general import SUF_BUTTON_OK +from sampletones_application.ui.elements.button import GUIButton +from sampletones_application.ui.elements.dialog import GUIDialogWindow +from sampletones_application.utils.gui.dialog_navigation import FocusStop +from sampletones_application.utils.gui.keyboard import KeyRouter +from sampletones_application.utils.gui.shortcuts.source import ShortcutSource +from sampletones_shared.types.callback import StringCallback + + +class GUINoticeWindow(GUIDialogWindow): + """A dialog stating something the reader acknowledges, with one button to dismiss it. + + What the notice says is drawn by the caller into the window's own tag, so a message, a + path or a list of settings all reach the reader through one dialog. A notice claiming + the screen answers to Tab, Enter and Escape; one reporting something already done stands + beside an interface that stays live, where the keys keep reaching what the reader was + working in and the button is clicked. + """ + + def __init__( + self, + tag: str, + *, + geometry: DialogGeometry, + title: str, + content: StringCallback, + ok_label: str, + key_router: KeyRouter, + shortcut_source: ShortcutSource, + claims_the_screen: bool, + ) -> None: + self._title = title + self._content = content + self._ok_label = ok_label + self._claims_the_screen = claims_the_screen + + super().__init__( + tag, + geometry, + key_router=key_router, + shortcut_source=shortcut_source, + ) + + def prepare(self, *_args: Any, **_kwargs: Any) -> None: + """What the notice says was named when it was built.""" + + def create_window(self) -> None: + ok_button_tag = compose_tag(self.tag, SUF_BUTTON_OK) + with self.dialog_window(label=self._title, on_close=self.hide): + self._content(self.tag) + dpg.add_separator(parent=self.tag) + GUIButton( + tag=ok_button_tag, + label=self._ok_label, + callback=self.hide, + parent=self.tag, + width=-1, + ) + + if self._claims_the_screen: + self._install_navigation( + [FocusStop.button(ok_button_tag, self.hide)], + on_escape=self.hide, + ) + + +def show_notice( + tag: str, + title: str, + content: StringCallback, + *, + geometry: DialogGeometry, + ok_label: str, + key_router: KeyRouter, + shortcut_source: ShortcutSource, + claims_the_screen: bool = True, +) -> None: + """Raises one notice the reader acknowledges, drawn at the size ``geometry`` states.""" + GUINoticeWindow( + tag=tag, + geometry=geometry, + title=title, + content=content, + ok_label=ok_label, + key_router=key_router, + shortcut_source=shortcut_source, + claims_the_screen=claims_the_screen, + ).show() diff --git a/src/sampletones_application/utils/gui/dialogs/windows/save_confirmation.py b/src/sampletones_application/utils/gui/dialogs/windows/save_confirmation.py index f100c6e3c..0f59b03d1 100644 --- a/src/sampletones_application/utils/gui/dialogs/windows/save_confirmation.py +++ b/src/sampletones_application/utils/gui/dialogs/windows/save_confirmation.py @@ -2,6 +2,7 @@ import dearpygui.dearpygui as dpg +from sampletones_application.layout.primitives import DialogGeometry from sampletones_application.tags.compose import compose_tag from sampletones_application.tags.general import ( SUF_BUTTON_CANCEL, @@ -30,14 +31,11 @@ class GUISaveConfirmationWindow(GUIDialogWindow): the prompt. """ - _fits_content = True - def __init__( self, tag: str, *, - width: int, - height: int, + geometry: DialogGeometry, wrap: int, save_label: str, cancel_label: str, @@ -56,8 +54,7 @@ def __init__( super().__init__( tag, - width, - height, + geometry, key_router=key_router, shortcut_source=shortcut_source, ) diff --git a/src/sampletones_application/utils/placement.py b/src/sampletones_application/utils/placement.py new file mode 100644 index 000000000..6ad940d7a --- /dev/null +++ b/src/sampletones_application/utils/placement.py @@ -0,0 +1,24 @@ +from typing import Tuple + + +def centered_position( + center: Tuple[int, int], + width: int, + height: int, +) -> Tuple[int, int]: + """Where a box of this size rests with its middle on ``center``. + + Each axis is held at zero at the least, so a box taller or wider than the space it is + centered in keeps its top-left corner reachable — which is what leaves a dialog's title bar + on screen when the reader has made the window smaller than the dialog it raises. + + Args: + center: The point the box is centered on. + width: The box's width. + height: The box's height. + + Returns: + Tuple[int, int]: The box's top-left corner. + """ + center_x, center_y = center + return max(0, round(center_x - width / 2)), max(0, round(center_y - height / 2)) diff --git a/src/sampletones_application/view_model/reconstruction/envelopes.py b/src/sampletones_application/view_model/reconstruction/envelopes.py new file mode 100644 index 000000000..cafbbbf4a --- /dev/null +++ b/src/sampletones_application/view_model/reconstruction/envelopes.py @@ -0,0 +1,30 @@ +from typing import Dict + +from pydantic import BaseModel + +from sampletones_application.view_model.shared.ownership import OwnershipLaneViewModel +from sampletones_core.constants.enums import ChannelName +from sampletones_core.exporters import Features + + +class ChannelEnvelopesViewModel(BaseModel, extra="forbid", frozen=True): + """What the instruments panel plots: each channel's envelopes and who holds their frames. + + The two are read from one part of the document, frame for frame, so a stretch painted under + a dimension's bars stands under the frames those bars draw. A voice written by hand answers + to no recording, and so does a document holding one, so both carry envelopes alone. + + Attributes: + channels: The envelopes each channel plots. + ownership: The recordings behind each channel's frames. + """ + + channels: Dict[ChannelName, Features] + ownership: Dict[ChannelName, OwnershipLaneViewModel] + + def __getitem__(self, channel_name: ChannelName) -> Features: + return self.channels[channel_name] + + def lane(self, channel_name: ChannelName) -> OwnershipLaneViewModel: + """The stretches under one channel's bars, empty where no recording is told apart there.""" + return self.ownership.get(channel_name, OwnershipLaneViewModel(channel_name=channel_name, runs=())) diff --git a/src/sampletones_application/view_model/shared/ownership.py b/src/sampletones_application/view_model/shared/ownership.py index dba95b435..facf64194 100644 --- a/src/sampletones_application/view_model/shared/ownership.py +++ b/src/sampletones_application/view_model/shared/ownership.py @@ -13,12 +13,14 @@ class OwnershipRunViewModel(BaseModel, extra="forbid", frozen=True): end_frame: The frame the stretch runs up to, one past its last. stem_id: The stem holding it, which names the color. position: Where that stem's entry stands on the record, which picks the color. + heard: Whether the reader hears that recording here, which settles how solidly it paints. """ start_frame: int end_frame: int stem_id: int position: int + heard: bool class OwnershipLaneViewModel(BaseModel, extra="forbid", frozen=True): @@ -27,6 +29,19 @@ class OwnershipLaneViewModel(BaseModel, extra="forbid", frozen=True): channel_name: ChannelName runs: Tuple[OwnershipRunViewModel, ...] + def up_to(self, frame_count: int) -> Tuple[OwnershipRunViewModel, ...]: + """The lane's runs over the first ``frame_count`` frames, the last of them ending there. + + A surface draws a reading as far as that reading goes, and the channel holds one lane for + every reading taken from it, so a surface drawing fewer frames than the channel holds + asks the lane for the stretches standing under them. + """ + return tuple( + run if run.end_frame <= frame_count else run.model_copy(update={"end_frame": frame_count}) + for run in self.runs + if run.start_frame < frame_count + ) + class OwnershipRibbonViewModel(BaseModel, extra="forbid", frozen=True): """The recordings behind each stretch of what a reader is listening to. diff --git a/src/sampletones_config/lang/en.yaml b/src/sampletones_config/lang/en.yaml index b3a975262..4766db971 100644 --- a/src/sampletones_config/lang/en.yaml +++ b/src/sampletones_config/lang/en.yaml @@ -15,6 +15,7 @@ global.dialog.label.remove: "Remove" global.dialog.label.change_and_retune: "Change and retune" global.dialog.label.dont_ask_again: "Don't ask again" global.dialog.label.add_anyway: "Add anyway" +global.dialog.label.unknown_duration: "?" # Global — Dialog titles global.dialog.title.main_window: "SampleToNES" @@ -362,7 +363,7 @@ main.source.label.drive: "drive" main.source.label.drive_mixed: "mixed" main.source.label.channel_cap: "Channels at once" main.source.tooltip.tooltip_new_recordings: "Every recording you add starts with these settings.\nPick a row in the Converter to change one recording or folder." -main.source.tooltip.tooltip_drive: "How hard this channel is driven during instruction selection and output.\nAt 1.00 the level is calibrated; higher values push the selection harder, adding a distortion-like effect." +main.source.tooltip.tooltip_drive: "How hard this channel is driven while a recording is converted.\nAt 1.00 the level is calibrated; higher values reach for louder instructions, adding a distortion-like effect." main.source.tooltip.tooltip_channel_cap: "How many of its channels a recording may sound in one frame,\nup to the channels it uses." main.source.message.status_channel_cap: "Sound at most {count} channels in one frame." @@ -393,7 +394,7 @@ main.converter.message.status_canceling: "Aborting the conversion..." main.converter.message.stage_loading: "reading the recordings" main.converter.message.stage_matching: "matching frames" main.converter.message.stage_decoding: "reading the channels" -main.converter.message.stage_rendering: "rendering the frames" +main.converter.message.stage_gathering: "gathering the frames" main.converter.message.status_canceled: "Conversion canceled." main.converter.message.status_input_label: "Input:" main.converter.message.status_output_label: "Destination:" diff --git a/src/sampletones_config/layout/general/colors.yaml b/src/sampletones_config/layout/general/colors.yaml index 898cd716f..fbfd9bb79 100644 --- a/src/sampletones_config/layout/general/colors.yaml +++ b/src/sampletones_config/layout/general/colors.yaml @@ -38,3 +38,4 @@ stems: - .stem_8 authored: .stem_authored rest: .stem_rest + left_out_fraction: 0.45 diff --git a/src/sampletones_config/layout/general/dialogs.yaml b/src/sampletones_config/layout/general/dialogs.yaml index 48e41eaba..3ede5f4f5 100644 --- a/src/sampletones_config/layout/general/dialogs.yaml +++ b/src/sampletones_config/layout/general/dialogs.yaml @@ -8,14 +8,12 @@ recovery: width: 640 height: 120 confirmation: + width: 420 height: 100 -text_input: - height: 104 -traceback: - width: 0 - height: 400 +traceback_height: 400 about: - width: 480 - height: 210 + window: + width: 480 + height: 210 logo: 56 padding: 40 diff --git a/src/sampletones_config/layout/graphs/bar_plot.yaml b/src/sampletones_config/layout/graphs/bar_plot.yaml index 2b6498e81..e7199004c 100644 --- a/src/sampletones_config/layout/graphs/bar_plot.yaml +++ b/src/sampletones_config/layout/graphs/bar_plot.yaml @@ -4,3 +4,4 @@ max_y: 100.0 bar_weight: 0.8 hover_alpha: 80 minimum_span: 2.0 +ownership_band: 0.12 diff --git a/src/sampletones_config/layout/project_properties/window.yaml b/src/sampletones_config/layout/project_properties/window.yaml index 4ba986957..f783886f5 100644 --- a/src/sampletones_config/layout/project_properties/window.yaml +++ b/src/sampletones_config/layout/project_properties/window.yaml @@ -1,2 +1 @@ width: 500 -height: 0 diff --git a/src/sampletones_config/layout/settings/audio.yaml b/src/sampletones_config/layout/settings/audio.yaml index 3bed24006..c279aba2b 100644 --- a/src/sampletones_config/layout/settings/audio.yaml +++ b/src/sampletones_config/layout/settings/audio.yaml @@ -1,6 +1,5 @@ window: width: 600 - height: 0 master_gain: slider_width: -65 label_color: .text_default diff --git a/src/sampletones_config/layout/settings/display.yaml b/src/sampletones_config/layout/settings/display.yaml index 3b53ce894..f873132fb 100644 --- a/src/sampletones_config/layout/settings/display.yaml +++ b/src/sampletones_config/layout/settings/display.yaml @@ -1,6 +1,4 @@ window: width: 460 - height: 0 countdown: width: 360 - height: 0 diff --git a/src/sampletones_config/layout/settings/keybindings.yaml b/src/sampletones_config/layout/settings/keybindings.yaml index 165bfc38d..2a4fc14e6 100644 --- a/src/sampletones_config/layout/settings/keybindings.yaml +++ b/src/sampletones_config/layout/settings/keybindings.yaml @@ -1,5 +1,4 @@ window: width: 620 - height: 0 list_height: 420 action_width: 320 diff --git a/src/sampletones_config/layout/settings/nsf.yaml b/src/sampletones_config/layout/settings/nsf.yaml index 5f7df9c30..19fca9030 100644 --- a/src/sampletones_config/layout/settings/nsf.yaml +++ b/src/sampletones_config/layout/settings/nsf.yaml @@ -1,5 +1,4 @@ window: width: 540 - height: 0 text_width: -64 frame_width: 40 diff --git a/src/sampletones_config/layout/settings/render.yaml b/src/sampletones_config/layout/settings/render.yaml index 3703a5caf..2165ce973 100644 --- a/src/sampletones_config/layout/settings/render.yaml +++ b/src/sampletones_config/layout/settings/render.yaml @@ -1,3 +1,2 @@ window: width: 540 - height: 0 diff --git a/src/sampletones_core/constants/enums.py b/src/sampletones_core/constants/enums.py index cb1006f41..c23a12ee5 100644 --- a/src/sampletones_core/constants/enums.py +++ b/src/sampletones_core/constants/enums.py @@ -91,14 +91,15 @@ class CQTWindow(StrEnum): ALL_CHANNELS: Final[FrozenSet[ChannelName]] = frozenset(ChannelName.items()) -TONE_CHANNELS: Final[FrozenSet[ChannelName]] = frozenset( +PULSE_CHANNELS: Final[FrozenSet[ChannelName]] = frozenset( { ChannelName.PULSE1, ChannelName.PULSE2, - ChannelName.TRIANGLE, } ) +TONE_CHANNELS: Final[FrozenSet[ChannelName]] = PULSE_CHANNELS | {ChannelName.TRIANGLE} + CHANNEL_ABBREVIATIONS: Final[Dict[ChannelName, Literal["P", "p", "T", "N"]]] = { ChannelName.PULSE1: "P", diff --git a/src/sampletones_core/data/document.py b/src/sampletones_core/data/document.py new file mode 100644 index 000000000..1bfc3d737 --- /dev/null +++ b/src/sampletones_core/data/document.py @@ -0,0 +1,78 @@ +import gzip +from contextlib import contextmanager +from typing import Final, Iterator, Protocol + +from sampletones_shared.types.path import Pathlike + +DOCUMENT_MAGIC: Final[bytes] = b"\x1f\x8b" +COMPRESSION_LEVEL: Final[int] = 9 +STATED_TIMESTAMP: Final[int] = 0 + + +class ByteStream(Protocol): + """A source bytes are read from a piece at a time, which is all a streaming read asks of one.""" + + def read(self, size: int = ...) -> bytes: + """The next ``size`` bytes, or the rest of the stream where the read names no size.""" + + +def compress_document(payload: bytes) -> bytes: + """The bytes a stored document is written as, its payload deflated. + + A document states its fields as names spelled out once per record, which is most of what a + file of thousands of records holds, so the payload deflates to a fraction of its size. The + bytes carry the deflate format's own magic, which tells a stored document apart from the + payload of one written before this framing. + + The timestamp the framing carries is stated rather than taken from the clock, so saving one + document twice writes the same bytes both times. + + Args: + payload: The document's serialized payload. + + Returns: + bytes: The bytes to store. + """ + return gzip.compress(payload, compresslevel=COMPRESSION_LEVEL, mtime=STATED_TIMESTAMP) + + +def decompress_document(stored: bytes) -> bytes: + """The payload a stored document holds. + + A document written before this framing carries its payload as it stands, and reads that way. + + Args: + stored: The bytes read from the document. + + Returns: + bytes: The document's serialized payload. + """ + if stored.startswith(DOCUMENT_MAGIC): + return gzip.decompress(stored) + + return stored + + +@contextmanager +def open_document(path: Pathlike) -> Iterator[ByteStream]: + """Opens the document at ``path`` as a stream over the payload it holds. + + A read that ends at the front of a document reads the front of the file, which is what lets a + library state its header ahead of its entries and be read by that header alone. + + Args: + path: The stored document. + + Yields: + ByteStream: The payload, from its first byte. + """ + with open(path, "rb") as file: + if file.read(len(DOCUMENT_MAGIC)) == DOCUMENT_MAGIC: + file.seek(0) + with gzip.GzipFile(fileobj=file, mode="rb") as stream: + yield stream + + return + + file.seek(0) + yield file diff --git a/src/sampletones_core/data/model.py b/src/sampletones_core/data/model.py index 41e6a36d8..b489e0626 100644 --- a/src/sampletones_core/data/model.py +++ b/src/sampletones_core/data/model.py @@ -21,6 +21,7 @@ import numpy as np from pydantic import BaseModel +from sampletones_core.data.document import compress_document, decompress_document from sampletones_shared.array import to_numpy from sampletones_shared.exceptions import ( DeserializationError, @@ -77,11 +78,11 @@ def deserialize( return cls.deserialize_inner(data, validation, fast=fast) def save(self, path: Pathlike) -> None: - save_binary(path, self.serialize()) + save_binary(path, compress_document(self.serialize())) @classmethod def load(cls, path: Pathlike, fast: bool = True) -> Self: - return cls.deserialize(load_binary(path), fast=fast) + return cls.deserialize(decompress_document(load_binary(path)), fast=fast) @classmethod def _construct(cls, fast: bool = True, **data: Any) -> Self: diff --git a/src/sampletones_core/data/stored.py b/src/sampletones_core/data/stored.py index c6e3093bc..a2ad7495f 100644 --- a/src/sampletones_core/data/stored.py +++ b/src/sampletones_core/data/stored.py @@ -1,7 +1,10 @@ +import gzip +import zlib from typing import FrozenSet import msgpack +from sampletones_core.data.document import open_document from sampletones_shared.types.data import SerializedData from sampletones_shared.types.path import Pathlike @@ -10,10 +13,11 @@ def read_leading_fields(path: Pathlike, names: FrozenSet[str]) -> SerializedData """The named top-level fields of a stored document, read from the front of its file. A stored :class:`DataModel` is one map whose fields follow the model's declaration order, so - the fields a model declares first sit in the first bytes of the file. The read decodes those + the fields a model declares first sit in the first bytes of its payload. The read decodes those fields and steps over any other in its way, and it ends once every named field is found, which - keeps it to the front of the file however large the rest is. A file whose front decodes as no - map reads as holding none of the fields. + keeps it to the front of the payload however large the rest is. A payload whose front decodes + as no map reads as holding none of the fields, a file whose framing the read cannot follow + included. Args: path: The stored document. @@ -23,8 +27,8 @@ def read_leading_fields(path: Pathlike, names: FrozenSet[str]) -> SerializedData SerializedData: Each named field the file holds, as stored. """ found: SerializedData = {} - with open(path, "rb") as file: - unpacker = msgpack.Unpacker(file, raw=False) + with open_document(path) as payload: + unpacker = msgpack.Unpacker(payload, raw=False) try: for _ in range(unpacker.read_map_header()): name = unpacker.unpack() @@ -35,7 +39,7 @@ def read_leading_fields(path: Pathlike, names: FrozenSet[str]) -> SerializedData found[name] = unpacker.unpack() if len(found) == len(names): break - except (ValueError, msgpack.OutOfData): + except (ValueError, msgpack.OutOfData, EOFError, gzip.BadGzipFile, zlib.error): return {} return found diff --git a/src/sampletones_core/library/data.py b/src/sampletones_core/library/data.py index c62d07e63..7e0b390a4 100644 --- a/src/sampletones_core/library/data.py +++ b/src/sampletones_core/library/data.py @@ -9,6 +9,7 @@ from sampletones_core.configs import Config, InstructionsLibraryConfig from sampletones_core.constants.enums import GeneratorClassName from sampletones_core.data import DataModel, Metadata, MetadataContract +from sampletones_core.data.document import decompress_document from sampletones_core.generators import GeneratorClassNames from sampletones_core.instructions import InstructionUnion from sampletones_shared.application import SAMPLETONES_LIBRARY_DATA_VERSION @@ -117,7 +118,7 @@ def load(cls, path: Pathlike, fast: bool = True) -> InstructionLibraryData: try: return InstructionLibraryData.deserialize( - binary, + decompress_document(binary), validation=cls.validate_metadata, fast=fast, ) diff --git a/src/sampletones_core/parallelization/progress.py b/src/sampletones_core/parallelization/progress.py index 1efb37ac8..7fb426697 100644 --- a/src/sampletones_core/parallelization/progress.py +++ b/src/sampletones_core/parallelization/progress.py @@ -2,10 +2,7 @@ from time import monotonic from typing import Deque, Final, Optional, Tuple -from sampletones_shared.utils.time import format_span - ESTIMATION_MEASUREMENTS_SAMPLES: Final[float] = 0.05 -UNKNOWN_DURATION: Final[str] = "?" class ETAEstimator: @@ -38,15 +35,6 @@ def update(self, completed_items: float) -> Optional[float]: return self._estimate_remaining_seconds(completed_items, now) - @classmethod - def format_duration(cls, seconds: Optional[float]) -> str: - """An estimate's remaining span, or the mark standing for one not yet measurable. - - A run needs two measurements before it has a rate, so the first moments of one answer - with the mark rather than with a figure. - """ - return UNKNOWN_DURATION if seconds is None else format_span(seconds) - def _get_estimation_measurements_samples(self, ems: float) -> int: if isinstance(ems, float): ems = round(ems * self._total) diff --git a/src/sampletones_core/reconstructions/reconstruction/reconstruction.py b/src/sampletones_core/reconstructions/reconstruction/reconstruction.py index 982dabf9a..e7f33f932 100644 --- a/src/sampletones_core/reconstructions/reconstruction/reconstruction.py +++ b/src/sampletones_core/reconstructions/reconstruction/reconstruction.py @@ -28,6 +28,7 @@ from sampletones_core.constants.algorithm import RESTING_STEM_ID from sampletones_core.constants.enums import ChannelName, FeatureKey from sampletones_core.data import DataModel, Metadata, MetadataContract +from sampletones_core.data.document import decompress_document from sampletones_core.exporters import ( CHANNEL_TO_EXPORTER_MAP, INSTRUCTION_TO_EXPORTER_MAP, @@ -35,9 +36,9 @@ ExporterUnion, Features, ) +from sampletones_core.generators.render import render_channels from sampletones_core.instructions import InstructionUnion from sampletones_core.reconstructions.reconstruction.instructions import InstructionsItem -from sampletones_core.reconstructions.reconstruction.rendering import render_streams from sampletones_core.reconstructions.reconstruction.stems.channel_assignment import ChannelAssignment from sampletones_core.reconstructions.reconstruction.stems.data import StemsData from sampletones_core.reconstructions.reconstruction.stems.filter import heard_instructions @@ -162,14 +163,14 @@ def audio_filepath(self) -> Tuple[Path, ...]: @cached_property def approximations(self) -> Dict[ChannelName, np.ndarray]: - """The audio each channel in play renders, at the drive its owner gives each frame. + """The audio each channel in play renders from the instructions it carries. - A reconstruction records the instructions a channel plays and the recording behind each - of its frames, so its sound is read from those rather than carried beside them. Reading - it here keeps one answer for the waveform, playback, an export and the mixed - approximation, and keeps a stored document to what it describes. + A reconstruction records the instructions a channel plays, so its sound is read from + those rather than carried beside them. Reading it here keeps one answer for the + waveform, playback, an export and the mixed approximation, and keeps a stored document + to what it describes. """ - return render_streams(self.instructions, self.stems_data, self.config) + return render_channels(self.instructions, self.config) @cached_property def approximation(self) -> np.ndarray: @@ -205,11 +206,11 @@ def initial_pitches(self) -> Dict[ChannelName, int]: @cached_property def held_features(self) -> Dict[ChannelName, Tuple[FeatureKey, ...]]: - """The dimensions each channel's instrument writes for itself. + """The dimensions each channel governs, whose envelopes an export leaves empty. - An instruction states every dimension of its frame, so which of them the instrument - itself writes is stated here: the rest are the channel's, and an export leaves their - envelopes empty for the player to fill from the value it holds. + An instrument writes the dimensions it describes and leaves the rest to the channel, + which keeps the value it already holds for as long as the instrument sounds. These + are the dimensions it leaves. """ return {channel_name: tuple(item.held_features) for channel_name, item in self.streams.items()} @@ -436,9 +437,8 @@ def _invalidate_derived_caches(reconstruction: Reconstruction) -> None: @classmethod def load(cls, path: Pathlike, fast: bool = True) -> Reconstruction: - binary = load_binary(path) return cls.deserialize_data( - binary, + load_binary(path), source=Path(path), validation=cls.validate_metadata, fast=fast, @@ -453,7 +453,7 @@ def deserialize_data( fast: bool = True, ) -> Reconstruction: try: - binary = upgrade_binary(ObjectKind.RECONSTRUCTION, binary) + binary = upgrade_binary(ObjectKind.RECONSTRUCTION, decompress_document(binary)) return cls.deserialize(binary, validation=validation, fast=fast) except (ValidationError, TypeError, ValueError, struct.error, IndexError) as exception: raise InvalidReconstructionValuesError( diff --git a/src/sampletones_core/reconstructions/reconstruction/rendering.py b/src/sampletones_core/reconstructions/reconstruction/rendering.py deleted file mode 100644 index ebd1ce3b3..000000000 --- a/src/sampletones_core/reconstructions/reconstruction/rendering.py +++ /dev/null @@ -1,107 +0,0 @@ -from typing import Dict, Mapping, Sequence - -import numpy as np - -from sampletones_core.configs import Config -from sampletones_core.constants.algorithm import UNIT_DRIVE -from sampletones_core.constants.enums import ChannelName -from sampletones_core.generators.render import render_instructions -from sampletones_core.instructions import InstructionUnion -from sampletones_core.reconstructions.reconstruction.stems.data import StemsData - -StemDrives = Mapping[int, Mapping[ChannelName, float]] - - -def frame_drive( - drives: StemDrives, - stem_id: int, - channel_name: ChannelName, -) -> float: - """The drive the stem owning a frame gives the channel. - - A frame no recording holds plays at unit drive, which is the level its silence stands at and - the level a frame the reader wrote sounds at. A recording that states a drive for a channel - plays it at that drive, and one that states none plays it as the library is calibrated, - which is the reading a setup written before drives existed carries. - - Args: - drives: The drive each recorded stem gives each channel it occupies. - stem_id: The stem holding the frame. - channel_name: The channel the frame belongs to. - - Returns: - float: The factor the frame's rendering is scaled by. - """ - return drives.get(stem_id, {}).get(channel_name, UNIT_DRIVE) - - -def stem_drives(stems_data: StemsData) -> Dict[int, Dict[ChannelName, float]]: - """The drive each recorded stem gives each channel it occupies, keyed by stem id.""" - return {entry.id: dict(entry.settings.drives) for entry in stems_data.config.entries} - - -def render_stream( - instructions: Sequence[InstructionUnion], - stem_ids: Sequence[int], - channel_name: ChannelName, - config: Config, - drives: StemDrives, -) -> np.ndarray: - """One channel's audio: its whole stream rendered, each frame at its owner's drive. - - The stream is rendered end to end so each frame continues the oscillator the one before it - left running, and the drive is applied to the samples a frame already rendered, so the level - a recording is pushed at reaches the mix without moving the waveform underneath it. - - Args: - instructions: The channel's instructions, one per frame. - stem_ids: The stem holding each of those frames. - channel_name: The channel the stream drives. - config: The configuration the frames are rendered at. - drives: The drive each recorded stem gives each channel it occupies. - - Returns: - np.ndarray: The channel's waveform, one frame per instruction. - """ - rendered = render_instructions(instructions, channel_name, config) - frame_length = config.library.frame_length - for position, stem_id in enumerate(stem_ids): - drive = frame_drive(drives, stem_id, channel_name) - if drive != UNIT_DRIVE: - rendered[position * frame_length : (position + 1) * frame_length] *= drive - - return rendered - - -def render_streams( - instructions: Mapping[ChannelName, Sequence[InstructionUnion]], - stems_data: StemsData, - config: Config, -) -> Dict[ChannelName, np.ndarray]: - """The audio of every channel that describes a frame, in channel order. - - This is the one reading a reconstruction's sound comes from: the waveform a reader sees, - what playback sends to the device, what an export writes and what the mixed approximation - sums. A channel standing by describes no frame and answers with none. - - Args: - instructions: The instructions each channel is driven by. - stems_data: The record naming the stem holding each frame. - config: The configuration the frames are rendered at. - - Returns: - Dict[ChannelName, np.ndarray]: The waveform each sounding channel renders to. - """ - drives = stem_drives(stems_data) - owners = stems_data.assignments_by_channel - return { - channel_name: render_stream( - instructions[channel_name], - owners.get(channel_name, ()), - channel_name, - config, - drives, - ) - for channel_name in ChannelName.items() - if instructions.get(channel_name) - } diff --git a/src/sampletones_core/reconstructions/reconstruction/stems/filter.py b/src/sampletones_core/reconstructions/reconstruction/stems/filter.py index f9be52e4f..68253593c 100644 --- a/src/sampletones_core/reconstructions/reconstruction/stems/filter.py +++ b/src/sampletones_core/reconstructions/reconstruction/stems/filter.py @@ -46,8 +46,9 @@ def heard_instructions( A frame held by a recording the reader left out reads as its channel's silent instruction at the index it stands on, so the reading lines up with the audio and the record frame for - frame. The reading ends where it last sounds, so a channel every recording is left out on - reads as standing by, and what it costs and what an export writes follow what is heard. + frame. The reading ends at the last frame the reader hears, and a channel whose sound the + reader's choice took away reads as standing by — which is the count an export writes and a + footprint measures. Args: stems_data: The record naming the stem holding each frame. @@ -60,23 +61,42 @@ def heard_instructions( heard: Dict[ChannelName, List[InstructionUnion]] = {} for channel_name, stream in instructions.items(): stem_ids = stems_data.assignments_by_channel.get(channel_name, ()) - masked = _masked_stream(stream, stem_ids, selection.stems_for(channel_name)) - heard[channel_name] = masked[: _sounding_length(masked)] + hearing = selection.stems_for(channel_name) + heard[channel_name] = _heard_stream(stream, stem_ids, hearing) return heard -def _masked_stream( +def _heard_stream( stream: Sequence[InstructionUnion], stem_ids: Sequence[int], heard: AbstractSet[int], ) -> List[InstructionUnion]: - """The stream with every frame outside the reader's hearing stating silence in its place.""" + """The stream a reader hears: silence where a recording is left out, cut where hearing ends. + + A rest answers to no recording, so every reader hears it and it stands where it is written. + That keeps a channel written down to rests alone in play, and leaves a channel whose sound + the reader's choice took away standing by, however many rests it also holds. + """ if not stream: return list(stream) null: InstructionUnion = type(stream[0]).null_instruction() - return [instruction if _is_heard(stem_ids, frame, heard) else null for frame, instruction in enumerate(stream)] + masked: List[InstructionUnion] = [] + last_heard = 0 + sounds = False + for frame, instruction in enumerate(stream): + if _is_heard(stem_ids, frame, heard): + masked.append(instruction) + last_heard = frame + 1 + sounds = sounds or instruction.on + else: + masked.append(null) + + if sounds or not any(instruction.on for instruction in stream): + return masked[:last_heard] + + return [] def _is_heard(stem_ids: Sequence[int], frame: int, heard: AbstractSet[int]) -> bool: @@ -85,12 +105,3 @@ def _is_heard(stem_ids: Sequence[int], frame: int, heard: AbstractSet[int]) -> b return True return heard_frame(stem_ids[frame], heard) - - -def _sounding_length(stream: Sequence[InstructionUnion]) -> int: - """How far a stream runs to its last sounding frame.""" - for frame in reversed(range(len(stream))): - if stream[frame].on: - return frame + 1 - - return 0 diff --git a/src/sampletones_core/reconstructions/reconstructor/candidates.py b/src/sampletones_core/reconstructions/reconstructor/candidates.py index 2f18a23c4..5286708c4 100644 --- a/src/sampletones_core/reconstructions/reconstructor/candidates.py +++ b/src/sampletones_core/reconstructions/reconstructor/candidates.py @@ -51,7 +51,7 @@ class CandidateProvider: """ Serves the library's candidates in the forms the matching reads them in. - Every quantity stands at the level the library sample plays, which the matching scales by + Every quantity stands at the level the library sample plays, which the matching reads at the drive the channel is asked for, so one set of rows serves every drive a run holds. A candidate's power is read off its phase-averaged library feature once per library and kept per generator class, so every frame mixes the same rows. Its waveform and its moments are diff --git a/src/sampletones_core/reconstructions/reconstructor/contribution.py b/src/sampletones_core/reconstructions/reconstructor/contribution.py index 6aa431f52..c6bfc3c5b 100644 --- a/src/sampletones_core/reconstructions/reconstructor/contribution.py +++ b/src/sampletones_core/reconstructions/reconstructor/contribution.py @@ -16,7 +16,7 @@ class Contribution: sequence adds its mean level, with its spread about that mean as the variance. Attributes: - power: The candidate's power density per bin, at the drive it plays at. + power: The candidate's power density per bin, read at unit drive. expectation: The waveform the candidate is expected to render over the frame. variance: The per-sample variance about that waveform. """ diff --git a/src/sampletones_core/reconstructions/reconstructor/matching.py b/src/sampletones_core/reconstructions/reconstructor/matching.py index 379b27193..c828686d1 100644 --- a/src/sampletones_core/reconstructions/reconstructor/matching.py +++ b/src/sampletones_core/reconstructions/reconstructor/matching.py @@ -44,8 +44,9 @@ class FrameMatcher: Carries the matching machinery the stems assignment works from: the two-stage criterion scoring and the per-candidate contribution build. What the scoring produces is a column of alternatives on one scale, the channel's silence among them, which the assignment turns into - ownership and the decoder into a stream. Every candidate is measured at the drive the caller - asks for, so a channel is answered at the level its own recording is given. + ownership and the decoder into a stream. A drive lifts what a channel reaches for: every + candidate is read at unit drive, so the row fitting the target is the one sounding it at the + drive, and a channel is answered at the level its own recording is given. """ config: Config @@ -68,10 +69,11 @@ def score_column( """ Score one generator class's candidates in two stages, each added to ``mix``. - Every candidate plays at ``drive``: its waveform follows the drive and its power the - drive's square. Each candidate's power is added to the mix and the result ranked by the - spectral term, which compares phase-averaged powers and is therefore immune to how the - waveforms happen to be phased. The ``top_k`` best and the class's silent instruction + Every candidate is read at unit drive: its waveform divides by the drive and its power + by the drive's square, so a rising drive reaches for a louder row and settles on the + loudest the class holds. Each candidate's power is added to the mix and the result + ranked by the spectral term, which compares phase-averaged powers and is therefore + immune to how the waveforms happen to be phased. The ``top_k`` best and the class's silent instruction then receive the full cost of the frame with them sounding. A candidate whose frames repeat one waveform shape adds its rendering at its best phase against what the mix leaves of the target's waveform, or its library sample from the start when best-phase @@ -82,7 +84,7 @@ def score_column( target: Target fragment to match. generator: The generator whose class is scored. mix: What the stem's other picks sound in the frame. - drive: The level the channel plays its candidates at. + drive: The level the channel reaches for, which every candidate is read against. Returns: The shortlisted candidates with their frame costs, best first, silence ahead of an @@ -92,13 +94,13 @@ def score_column( ValueError: If the library lacks the class's silent instruction. """ candidates = self.candidate_provider.candidates({generator.class_name(): generator}) - driven = _driven(candidates.powers, drive) - mixed = self.candidate_provider.features_of(driven + xp.asarray(mix.power)) + unit_powers = _at_unit_drive(candidates.powers, drive) + mixed = self.candidate_provider.features_of(unit_powers + xp.asarray(mix.power)) spectral_costs = self.scorer.spectral_costs(target, mixed) shortlist = self._shortlist(spectral_costs, candidates, generator) residual = mix.residual_waveform(target) - powers = np.asarray(to_numpy(driven[xp.asarray(shortlist)]), dtype=np.float64) + powers = np.asarray(to_numpy(unit_powers[xp.asarray(shortlist)]), dtype=np.float64) contributions = [ self.contribution(candidates.instructions[index], residual, power, drive=drive) for index, power in zip(shortlist, powers) @@ -141,9 +143,9 @@ def contribution( Args: instruction: The candidate. residual: What the frame's waveform holds beyond the stem's other picks. - power: The candidate's power density per bin at ``drive``. - drive: The level the candidate plays at, which its waveform and its mean level - follow, and its variance the drive's square. + power: The candidate's power density per bin, read at unit drive. + drive: The level the channel reaches for, which the candidate's waveform and mean + level divide by, and its variance the drive's square. Returns: Contribution: The candidate's power, expected waveform and variance. @@ -156,14 +158,14 @@ def contribution( mean, variance = self.candidate_provider.expected_moments(instruction) return Contribution( power=power, - expectation=np.full(residual.shape[0], drive * mean), - variance=drive**2 * variance, + expectation=np.full(residual.shape[0], mean / drive), + variance=variance / drive**2, ) if self.config.generation.calculation.find_best_phase: waveform = self.phase_aligner.align(residual, instruction, drive) else: - waveform = drive * self.candidate_provider.library_waveform(instruction) + waveform = self.candidate_provider.library_waveform(instruction) / drive return Contribution(power=power, expectation=waveform, variance=0.0) @@ -190,12 +192,12 @@ def _shortlist( return shortlist -def _driven(powers: xp.ndarray, drive: float) -> xp.ndarray: - """The candidates' powers at ``drive``, which a power follows by the drive's square.""" +def _at_unit_drive(powers: xp.ndarray, drive: float) -> xp.ndarray: + """The candidates' powers read at unit drive, which a power divides by the drive's square.""" if drive == UNIT_DRIVE: return powers - return powers * drive**2 + return powers / drive**2 def column_of(scored: Column, width: int) -> Column: diff --git a/src/sampletones_core/reconstructions/reconstructor/phase.py b/src/sampletones_core/reconstructions/reconstructor/phase.py index 49abd239e..f6d94bbda 100644 --- a/src/sampletones_core/reconstructions/reconstructor/phase.py +++ b/src/sampletones_core/reconstructions/reconstructor/phase.py @@ -25,12 +25,12 @@ def __init__( @abstractmethod def align(self, waveform: np.ndarray, instruction: InstructionUnion, drive: float) -> np.ndarray: - """The frame the candidate renders at its best phase against ``waveform``, playing at ``drive``.""" + """The frame the candidate renders at its best phase against ``waveform``, read at unit drive.""" def _rendering(self, instruction: InstructionUnion, shift: int, drive: float) -> np.ndarray: - """The candidate's frame starting ``shift`` samples into its library sample, playing at ``drive``.""" + """The candidate's frame starting ``shift`` samples into its library sample, read at unit drive.""" fragment = self.library_data[instruction].get_fragment(shift, self.config, self.window) - return np.asarray(fragment.audio, dtype=np.float64) * drive + return np.asarray(fragment.audio, dtype=np.float64) / drive def _cycle(self, instruction: InstructionUnion) -> np.ndarray: """Two cycles of the candidate's library sample, which every shift of one frame reads from.""" @@ -40,13 +40,13 @@ def _cycle(self, instruction: InstructionUnion) -> np.ndarray: class SlidingRmsePhaseAligner(PhaseAligner): """ - Finds the cyclic shift minimizing the RMSE between the waveform and the drive-scaled - candidate, and returns the candidate's frame at that shift. + Finds the cyclic shift minimizing the RMSE between the waveform and the candidate read at + unit drive, and returns the candidate's frame at that shift. """ def align(self, waveform: np.ndarray, instruction: InstructionUnion, drive: float) -> np.ndarray: windows = sliding_window_view(self._cycle(instruction), self.config.library.frame_length) - remainder = np.asarray(waveform, dtype=np.float64) - drive * windows + remainder = np.asarray(waveform, dtype=np.float64) - windows / drive rmse = np.sqrt((remainder**2).mean(axis=1)) return self._rendering(instruction, int(np.argmin(rmse)), drive) @@ -54,10 +54,10 @@ def align(self, waveform: np.ndarray, instruction: InstructionUnion, drive: floa class CrossCorrelationPhaseAligner(PhaseAligner): """ - Finds the cyclic shift minimizing the squared error between the waveform and the - drive-scaled candidate via FFT cross-correlation, expanding - `||waveform - drive * candidate||^2` into a per-shift cost of - `drive * energy - 2 * correlation` (the constant waveform energy and the positive + Finds the cyclic shift minimizing the squared error between the waveform and the candidate + read at unit drive via FFT cross-correlation, expanding + `||waveform - candidate / drive||^2` into a per-shift cost of + `energy / drive - 2 * correlation` (the constant waveform energy and the positive drive factor drop out of the argmin), and returns the candidate's frame at that shift. The sliding energy of a candidate depends on its library sample alone, so it is read once per instruction. @@ -77,7 +77,7 @@ def align(self, waveform: np.ndarray, instruction: InstructionUnion, drive: floa target = np.asarray(waveform, dtype=np.float64) correlation = fftconvolve(cycle, target[::-1], mode="valid") - cost = drive * self._sliding_energy(instruction, cycle) - 2.0 * correlation + cost = self._sliding_energy(instruction, cycle) / drive - 2.0 * correlation return self._rendering(instruction, int(np.argmin(cost)), drive) def _sliding_energy(self, instruction: InstructionUnion, cycle: np.ndarray) -> np.ndarray: diff --git a/src/sampletones_core/reconstructions/reconstructor/reconstructor.py b/src/sampletones_core/reconstructions/reconstructor/reconstructor.py index 4a79da72f..06b4aa124 100644 --- a/src/sampletones_core/reconstructions/reconstructor/reconstructor.py +++ b/src/sampletones_core/reconstructions/reconstructor/reconstructor.py @@ -168,6 +168,7 @@ def reconstruct( assignment.release_silent(streams) self._drop_resting_channels(assignment, streams) streams = self._refiner(stems_config).refine(streams, assignment.stem_ids, prepared.recordings) + announce(report, ReconstructionStage.DECODING, WHOLE_STAGE, WHOLE_STAGE) self._record_streams(streams, report) return Reconstruction.from_state( self.state, @@ -319,11 +320,11 @@ def _record_streams( """ frames = self._frame_count(streams) for position in range(frames): - announce(report, ReconstructionStage.RENDERING, position, frames) + announce(report, ReconstructionStage.GATHERING, position, frames) for channel_name in self.state.channel_names: self.state.append(channel_name, streams[channel_name][position].instruction) - announce(report, ReconstructionStage.RENDERING, frames, frames) + announce(report, ReconstructionStage.GATHERING, frames, frames) @staticmethod def _frame_count(streams: Streams) -> int: diff --git a/src/sampletones_core/reconstructions/reconstructor/stems/assignment/columns.py b/src/sampletones_core/reconstructions/reconstructor/stems/assignment/columns.py index 15a266e93..08fbafd5c 100644 --- a/src/sampletones_core/reconstructions/reconstructor/stems/assignment/columns.py +++ b/src/sampletones_core/reconstructions/reconstructor/stems/assignment/columns.py @@ -11,7 +11,7 @@ class ColumnGroup: Attributes: generator: The generator of the lowest channel in the group, which stands for it. - drive: The level every channel of the group plays its candidates at. + drive: The level every channel of the group reaches for. """ generator: GeneratorUnion diff --git a/src/sampletones_core/reconstructions/reconstructor/stems/assignment/session.py b/src/sampletones_core/reconstructions/reconstructor/stems/assignment/session.py index bd0aa7af0..7874f1f6c 100644 --- a/src/sampletones_core/reconstructions/reconstructor/stems/assignment/session.py +++ b/src/sampletones_core/reconstructions/reconstructor/stems/assignment/session.py @@ -3,7 +3,11 @@ import numpy as np -from sampletones_core.constants.algorithm import RESTING_FRAME_COST, STEM_ACTIVITY_FLOOR +from sampletones_core.constants.algorithm import ( + RESTING_FRAME_COST, + STEM_ACTIVITY_FLOOR, + UNIT_DRIVE, +) from sampletones_core.constants.enums import ( ChannelName, GeneratorClassName, @@ -35,22 +39,25 @@ @dataclass(frozen=True) class StemOffer: - """What one stem offers for a channel: that channel's column in the stem's frame, and what its head saves. + """What one stem offers for a channel: that channel's column in the stem's frame, and what it saves. - The column arrives best first, so its head is the candidate the stem competes with. + The column arrives best first, so its head is the candidate the channel would play. Attributes: stem_id: The stem making the offer. generator: The generator standing for the channel the stem would take. column: The channel's candidates scored in the stem's frame at the drive the stem gives the channel, best first. - improvement: How far the head lowers the stem's frame cost, weighted by the energy of the - stem's frame. + unit_cost: The cost the channel's own best candidate at unit drive reaches in the stem's + frame, which the offer is ranked on and the stem stands at once the channel is taken. + improvement: How far the channel at unit drive lowers the stem's frame cost, weighted by + the energy of the stem's frame. """ stem_id: int generator: GeneratorUnion column: Column + unit_cost: float improvement: float @property @@ -81,11 +88,13 @@ class AssignmentSession: A stem's mix is what its picks sound in its own frame, so a candidate is scored by the cost that frame reaches with the candidate sounding beside them, and the channel's silence is one - of the candidates. Every candidate plays at the drive the stem gives the channel, so one + of the candidates. Every candidate is read at the drive the stem gives the channel, so one column answers the channels of a kind the stem drives alike and the channels it drives apart hold columns of their own. A channel is taken where it lowers a frame's cost, by the stem - whose sound it covers most. Only the free channels are shared: the stems compete for them, - ordered by the hierarchy, and the stems sounding in the frame are the ones that compete. + whose sound it covers most, measured at unit drive so a drive settles what a channel plays + and leaves the stems standing where they are. Only the free channels are shared: the stems + compete for them, ordered by the hierarchy, and the stems sounding in the frame are the ones + that compete. """ def __init__( @@ -183,18 +192,23 @@ def _best_offer(self, stem_ids: Sequence[int]) -> Optional[StemOffer]: A cost is a fraction of its own frame's energy, so two stems' costs stand on different scales; weighting a lowering by the energy behind it states it in absolute terms, so the - channel reaches the stem whose sound it covers most. An offer whose head is the channel's - silence, or which lowers nothing, stands aside. Equal offers leave the one already - standing, which is earlier in level order and then in channel order, so a rerun of the - same frame assigns the same way. + channel reaches the stem whose sound it covers most. The lowering is read from the + channel scored at unit drive while the driven column decides what the channel plays, so + the stems compete on how much of a frame a channel covers and a drive stays a property of + the sound its stem makes. An offer whose head is the channel's silence, or which lowers + nothing, stands aside. Equal offers leave the one already standing, which is earlier in + level order and then in channel order, so a rerun of the same frame assigns the same way. """ best: Optional[StemOffer] = None for stem_id in stem_ids: for group in column_groups(self._remaining_channels(stem_id), self._drives(stem_id)): column = self._column(stem_id, group.generator, group.drive) - head = column[0] - improvement = (self.frame_costs[stem_id] - head.cost) * self.energies[stem_id] - if not head.instruction.on or improvement <= 0.0: + if not column[0].instruction.on: + continue + + unit_cost = self._column(stem_id, group.generator, UNIT_DRIVE)[0].cost + improvement = (self.frame_costs[stem_id] - unit_cost) * self.energies[stem_id] + if improvement <= 0.0: continue if best is None or improvement > best.improvement: @@ -202,17 +216,22 @@ def _best_offer(self, stem_ids: Sequence[int]) -> Optional[StemOffer]: stem_id=stem_id, generator=group.generator, column=column, + unit_cost=unit_cost, improvement=improvement, ) return best def _take(self, offer: StemOffer) -> None: - """Gives the offer's channel to its stem, sounding the head in that stem's mix.""" + """Gives the offer's channel to its stem, sounding the head in that stem's mix. + + The stem stands at the cost the channel reaches at unit drive, which is the scale the + next offer is measured against, so one drive-free lowering follows another. + """ stem_id = offer.stem_id self._record(stem_id, ChannelName(offer.generator.name), offer.column) self.mixes[stem_id] = self.mixes[stem_id].added(offer.head.contribution) - self.frame_costs[stem_id] = offer.head.cost + self.frame_costs[stem_id] = offer.unit_cost self._forget_columns(stem_id) def _settle_declined(self) -> None: diff --git a/src/sampletones_core/reconstructions/stage.py b/src/sampletones_core/reconstructions/stage.py index 686ef53a9..349c16c20 100644 --- a/src/sampletones_core/reconstructions/stage.py +++ b/src/sampletones_core/reconstructions/stage.py @@ -6,26 +6,28 @@ class ReconstructionStage(StrEnum): """The work a reconstruction is in the middle of, as the progress it reports names it. A run reads its recordings onto one scale, matches every frame against the library, reads each - channel's frames into the stream it plays, and renders that stream back to the audio the - reconstruction carries. Each stage counts in its own unit, so what a report means is read from - the stage it names. + channel's frames into the stream it plays, and gathers those streams into the document it + answers with. Each stage counts in its own unit, so what a report means is read from the stage + it names. - Matching visits the library once per frame per stem and is what a run spends its time on, which - is what :data:`STAGE_WEIGHTS` states: the bulk of a reading belongs to matching so a bar tracks - the time a run actually takes, while the stages around it keep enough of it to move visibly as - they pass. The weights are approximations measured over whole runs, and matching earns a larger - share the longer the recording is, so the stages around it are given what they hold on a short - one — where a bar standing still is noticed. + Matching visits the library once per frame per stem and is the whole of what a run spends its + time on: measured over whole runs it takes some ninety-seven parts in a hundred, decoding two, + and the gathering a tenth of one. :data:`STAGE_WEIGHTS` turns those measurements into shares a + bar reads well. Loading takes eight, because a run opening a library of its own pays that + reading once and a short recording spends a quarter of itself on it — a bar standing still is + noticed, while one moving a little early is not. The gathering takes one so the last stage + moves visibly, decoding keeps the two it measures, and matching keeps the eighty-nine that + remain. - The weights are counts rather than fractions, so what a stage is worth is stated against the - others and a reading is one division at the point of use — which is what lets the last stage - arrive exactly at the whole run. + The weights are counts, each stating what a stage is worth against the others, and a reading + divides once at the point of use — which is what lets the last stage arrive exactly at the + whole run. """ LOADING = "loading" MATCHING = "matching" DECODING = "decoding" - RENDERING = "rendering" + GATHERING = "gathering" @property def weight(self) -> int: @@ -57,9 +59,9 @@ def offset(self) -> float: STAGE_WEIGHTS: Final[Mapping[ReconstructionStage, int]] = { ReconstructionStage.LOADING: 8, - ReconstructionStage.MATCHING: 82, + ReconstructionStage.MATCHING: 89, ReconstructionStage.DECODING: 2, - ReconstructionStage.RENDERING: 8, + ReconstructionStage.GATHERING: 1, } TOTAL_STAGE_WEIGHT: Final[int] = sum(STAGE_WEIGHTS.values()) diff --git a/src/sampletones_player/builder.py b/src/sampletones_player/builder.py index 39d9c2581..9c2bff749 100644 --- a/src/sampletones_player/builder.py +++ b/src/sampletones_player/builder.py @@ -129,7 +129,9 @@ def instructions_from_instruments( return instructions -def loop_tick_from_instruments(instruments: Sequence[InstrumentExport]) -> Optional[int]: +def loop_tick_from_instruments( + instruments: Sequence[InstrumentExport], +) -> Optional[int]: """The tick a request's song returns to once it ends. A song repeats from its first tick where every slice it carries repeats, and ends at its diff --git a/src/sampletones_player/clock/schedule.py b/src/sampletones_player/clock/schedule.py index 82ee27363..a7dc6d064 100644 --- a/src/sampletones_player/clock/schedule.py +++ b/src/sampletones_player/clock/schedule.py @@ -130,7 +130,10 @@ def fixed_point_step(self) -> FixedPointStep: Every answer the schedule gives counts in this step, so it is computed once and held. """ - whole, fraction = divmod(round(self.ticks_per_play_call * FIXED_POINT_SCALE), FIXED_POINT_SCALE) + whole, fraction = divmod( + round(self.ticks_per_play_call * FIXED_POINT_SCALE), + FIXED_POINT_SCALE, + ) return FixedPointStep(whole=whole, fraction=fraction) def maximum_drift(self, play_calls: int) -> int: diff --git a/src/sampletones_player/compression/absent.py b/src/sampletones_player/compression/absent.py deleted file mode 100644 index a333d2cb9..000000000 --- a/src/sampletones_player/compression/absent.py +++ /dev/null @@ -1,16 +0,0 @@ -from sampletones_player.specification.compression import INITIAL_PLANE_VALUE - - -def is_absent(plane: bytes) -> bool: - """Whether a plane plays only the value every plane starts at, so the block leaves it out. - - The driver starts every plane at that value and leaves an absent one standing there, which - is what makes a channel that never bends cost its bend plane nothing. - - Args: - plane: The values the plane plays. - - Returns: - bool: Whether the plane holds no other value. - """ - return set(plane) <= {INITIAL_PLANE_VALUE} diff --git a/src/sampletones_player/compression/admit.py b/src/sampletones_player/compression/admit.py index 9cca4fd97..027593174 100644 --- a/src/sampletones_player/compression/admit.py +++ b/src/sampletones_player/compression/admit.py @@ -1,13 +1,25 @@ from typing import Final, List, NamedTuple, Sequence, Set, Tuple -from sampletones_player.compression.dictionary.phrase import Phrase, phrase_entry_size -from sampletones_player.compression.dictionary.table import PhraseTable, phrase_table -from sampletones_player.compression.matches.cache import MIN_PHRASE_TICKS, MatchCache +from sampletones_player.compression.dictionary.phrase import ( + Phrase, + phrase_entry_size, +) +from sampletones_player.compression.dictionary.table import ( + PhraseTable, + phrase_table, +) +from sampletones_player.compression.matches.cache import ( + MIN_PHRASE_TICKS, + MatchCache, +) from sampletones_player.compression.matches.shift import NO_SHIFT, asked_shift from sampletones_player.compression.options import CodecOptions from sampletones_player.compression.parse.result import Parse from sampletones_player.compression.tokens.sizes import phrase_size -from sampletones_player.specification.compression import MAX_PHRASE_IDS, PHRASE_ID_ESCAPE +from sampletones_player.specification.compression import ( + MAX_PHRASE_IDS, + PHRASE_ID_ESCAPE, +) from sampletones_shared.logger import logger CROWDED_PHRASE_ID: Final[int] = PHRASE_ID_ESCAPE @@ -95,8 +107,8 @@ def _payment( if played == 0: return 0 - stated = phrase_size(CROWDED_PHRASE_ID, UNSHIFTED_TOKEN_TRANSPOSE) - shifted = phrase_size(CROWDED_PHRASE_ID, SHIFTED_TOKEN_TRANSPOSE) + stated = phrase_size(CROWDED_PHRASE_ID, UNSHIFTED_TOKEN_TRANSPOSE, default=False) + shifted = phrase_size(CROWDED_PHRASE_ID, SHIFTED_TOKEN_TRANSPOSE, default=False) spent = stated + shifted * (played - 1) + phrase_entry_size(phrase.length) return paid - spent diff --git a/src/sampletones_player/compression/compressed.py b/src/sampletones_player/compression/compressed.py index 74d4d9f31..eb7eb4788 100644 --- a/src/sampletones_player/compression/compressed.py +++ b/src/sampletones_player/compression/compressed.py @@ -1,11 +1,10 @@ from __future__ import annotations -from typing import Optional, Sequence, Tuple +from typing import Optional, Tuple from pydantic import BaseModel, ConfigDict, model_validator from sampletones_player.compression.dictionary.table import PhraseTable -from sampletones_player.compression.entries import stream_entry from sampletones_player.compression.planes.order import PlaneOrder @@ -16,6 +15,8 @@ class CompressedPlanes(BaseModel): phrases: The dictionary every stream's tokens name. streams: The tokens each plane is written as, in the order the song block writes them. ticks: The ticks the song lasts, which is where each stream stops being read. + loop_entries: The byte each stream is re-entered at when the song comes round, counted + from that stream's own start, ``None`` for a plane holding no stream. """ model_config = ConfigDict(extra="forbid", frozen=True) @@ -23,6 +24,7 @@ class CompressedPlanes(BaseModel): phrases: PhraseTable streams: PlaneOrder ticks: int + loop_entries: Tuple[Optional[int], ...] @model_validator(mode="after") def _validate_the_song_lasts(self) -> CompressedPlanes: @@ -31,30 +33,18 @@ def _validate_the_song_lasts(self) -> CompressedPlanes: return self + @model_validator(mode="after") + def _validate_every_stream_states_where_it_is_re_entered( + self, + ) -> CompressedPlanes: + if len(self.loop_entries) != len(self.streams): + raise ValueError( + f"an entry stands for each of the {len(self.streams)} streams, " f"and {len(self.loop_entries)} stand" + ) + + return self + @property def size(self) -> int: """The bytes the dictionary and every plane's stream take together.""" return self.phrases.size + sum(len(stream) for stream in self.streams) - - def entries(self, positions: Sequence[int]) -> Tuple[Optional[int], ...]: - """The byte each stream is re-entered at, each plane standing at a position of its own. - - A song returning to a tick re-enters every plane where that tick leaves it, which for a - bend plane is the count of flagged ticks before it — see ``SongPlanes.positions``. - - Args: - positions: Where each plane stands once the song returns, in the order the song block - writes them. - - Returns: - Tuple[Optional[int], ...]: One byte offset per plane, each counted from its own - stream's start, in the order the song block writes them; ``None`` for an absent - plane, which holds no stream to re-enter. - - Raises: - ValueError: If a stream spans its position rather than starting a token there. - """ - return tuple( - stream_entry(stream, position) if stream else None - for stream, position in zip(self.streams, positions, strict=True) - ) diff --git a/src/sampletones_player/compression/decode.py b/src/sampletones_player/compression/decode.py index 2dc0a8848..ef4ff72fe 100644 --- a/src/sampletones_player/compression/decode.py +++ b/src/sampletones_player/compression/decode.py @@ -1,18 +1,27 @@ -from typing import Tuple +from typing import List, Tuple from sampletones_player.compression.compressed import CompressedPlanes from sampletones_player.compression.dictionary.table import PhraseTable from sampletones_player.compression.planes.flags import flagged_ticks from sampletones_player.compression.planes.order import PlaneOrder from sampletones_player.compression.planes.song import SongPlanes +from sampletones_player.compression.planes.symbols import unpack_plane from sampletones_player.specification.binary import BYTE_VALUES from sampletones_player.specification.compression import ( - INITIAL_PLANE_VALUE, + DEFAULT_COUNT_FLAG, PHRASE_ID_ESCAPE, + PHRASE_ID_MASK, TOKEN_OPERAND_MASK, TOKEN_TAG_MASK, TokenTag, ) +from sampletones_player.specification.planes import ( + PLANES, + SINGLE_TICK, + Plane, + PlaneRole, + plane_index, +) def _phrase_values( @@ -23,13 +32,17 @@ def _phrase_values( *, transposed: bool, ) -> Tuple[bytes, int]: - phrase_id = operand - if operand == PHRASE_ID_ESCAPE: + phrase_id = operand & PHRASE_ID_MASK + if phrase_id == PHRASE_ID_ESCAPE: phrase_id = data[position] position += 1 - ticks = data[position] + 1 - position += 1 + if operand & DEFAULT_COUNT_FLAG: + ticks = table[phrase_id].default + else: + ticks = data[position] + 1 + position += 1 + transpose = 0 if transposed: transpose = data[position] @@ -41,28 +54,38 @@ def _phrase_values( return played, position -def decode_plane(data: bytes, table: PhraseTable, ticks: int) -> bytes: +def decode_plane( + data: bytes, + table: PhraseTable, + plane: Plane, + ticks: int, +) -> bytes: """Plays a plane's token stream back into the values it writes, tick by tick. This is the reading the driver performs, stated where it is testable: every encoding is held - against it, so what the console plays and what the encoder meant are the same values. An - absent plane's empty stream plays the value every plane starts at throughout. + against it, so what the console plays and what the encoder meant are the same values. A + symbol covers the ticks its own count states, so the reading stops once the ticks the song + lasts are covered, wherever in a symbol that falls. An absent plane's empty stream plays the + value the driver seeds it to throughout. Args: data: The plane's token stream. table: The dictionary the tokens name. + plane: The plane the stream belongs to, for the byte it seeds to and how its byte divides. ticks: The ticks the song lasts. Returns: bytes: The values the plane writes, one per tick. """ + form = plane.form if not data: - return bytes((INITIAL_PLANE_VALUE,)) * ticks + return bytes((plane.seeded,)) * ticks - values = bytearray() - current = INITIAL_PLANE_VALUE + symbols = bytearray() + current = form.symbol(plane.seeded, SINGLE_TICK) + covered = 0 position = 0 - while len(values) < ticks: + while covered < ticks: opcode = data[position] position += 1 operand = opcode & TOKEN_OPERAND_MASK @@ -90,24 +113,10 @@ def decode_plane(data: bytes, table: PhraseTable, ticks: int) -> bytes: ) current = played[-1] - values.extend(played) - - return bytes(values[:ticks]) + symbols.extend(played) + covered += sum(form.repeated(symbol) for symbol in played) - -def _tone_planes( - compressed: CompressedPlanes, - control: bytes, - value: bytes, - bend: bytes, -) -> Tuple[bytes, bytes, bytes]: - """A tone channel's planes played back, its bend plane as long as its value plane flags.""" - played = decode_plane(value, compressed.phrases, compressed.ticks) - return ( - decode_plane(control, compressed.phrases, compressed.ticks), - played, - decode_plane(bend, compressed.phrases, flagged_ticks(played)), - ) + return unpack_plane(bytes(symbols), form)[:ticks] def decode_planes(compressed: CompressedPlanes) -> SongPlanes: @@ -120,16 +129,15 @@ def decode_planes(compressed: CompressedPlanes) -> SongPlanes: compressed: The dictionary, the streams and the ticks the song lasts. Returns: - SongPlanes: The planes under the channel each belongs to. + SongPlanes: Every plane, in the order the song block writes them. """ - streams = compressed.streams - played = PlaneOrder.across( - ( - *_tone_planes(compressed, streams.pulse1_control, streams.pulse1_value, streams.pulse1_bend), - *_tone_planes(compressed, streams.pulse2_control, streams.pulse2_value, streams.pulse2_bend), - *_tone_planes(compressed, streams.triangle_control, streams.triangle_value, streams.triangle_bend), - decode_plane(streams.noise_control, compressed.phrases, compressed.ticks), - decode_plane(streams.noise_value, compressed.phrases, compressed.ticks), + played: List[bytes] = [] + for plane, stream in zip(PLANES, compressed.streams, strict=True): + reach = ( + flagged_ticks(played[plane_index(plane.channel, PlaneRole.VALUE)]) + if plane.spans_flagged_ticks + else compressed.ticks ) - ) - return SongPlanes.from_order(played) + played.append(decode_plane(stream, compressed.phrases, plane, reach)) + + return SongPlanes(planes=PlaneOrder.across(played)) diff --git a/src/sampletones_player/compression/dictionary/phrase.py b/src/sampletones_player/compression/dictionary/phrase.py index ea8fabe6d..72cbdc112 100644 --- a/src/sampletones_player/compression/dictionary/phrase.py +++ b/src/sampletones_player/compression/dictionary/phrase.py @@ -7,6 +7,8 @@ from sampletones_player.specification.binary import BYTE_VALUES from sampletones_player.specification.compression import ( MAX_PHRASE_LENGTH, + NO_DEFAULT_COUNT, + PHRASE_DEFAULT_SIZE, PHRASE_LENGTH_SIZE, PHRASE_TABLE_ENTRY_SIZE, ) @@ -24,7 +26,7 @@ def phrase_entry_size(length: int) -> int: Returns: int: The bytes the phrase and its table entry take together. """ - return PHRASE_TABLE_ENTRY_SIZE + PHRASE_LENGTH_SIZE + length + return PHRASE_TABLE_ENTRY_SIZE + PHRASE_LENGTH_SIZE + PHRASE_DEFAULT_SIZE + length class Phrase(BaseModel): @@ -35,11 +37,13 @@ class Phrase(BaseModel): Attributes: body: The values the phrase plays, one per tick. + default: The count its tokens play it at most often, which a token may leave unstated. """ model_config = ConfigDict(extra="forbid", frozen=True) body: bytes + default: int = NO_DEFAULT_COUNT @model_validator(mode="after") def _validate_the_body_fits_a_table_entry(self) -> Phrase: diff --git a/src/sampletones_player/compression/dictionary/prune.py b/src/sampletones_player/compression/dictionary/prune.py index 2b9e60254..50fce0792 100644 --- a/src/sampletones_player/compression/dictionary/prune.py +++ b/src/sampletones_player/compression/dictionary/prune.py @@ -24,21 +24,27 @@ def prune( table: PhraseTable, references: Mapping[int, int], savings: Mapping[int, int], + counts: Mapping[int, int], ) -> PhraseTable: """Rebuilds a table around the phrases that pay for themselves, most named first. A phrase earns its place by sparing the streams more bytes than its own entry takes, and the ids it competes for are worth a byte apiece: the cheap ones ride inside an opcode, so the - phrases named most often take them and the tokens naming those shed a byte each. + phrases named most often take them and the tokens naming those shed a byte each. Each phrase + kept carries the count its own tokens play it at most often, which is the count a token may + then leave unstated. Args: table: The table the tokens were parsed against. references: How many tokens name each phrase id. savings: The bytes each phrase's tokens spare the streams. + counts: The count each phrase's tokens play it at most often. Returns: PhraseTable: The phrases worth keeping, ordered by how often they are named. """ kept = _pays_for_itself(table, references, savings) kept.sort(key=lambda entry: (-entry.references, entry.phrase_id)) - return PhraseTable(phrases=tuple(table[entry.phrase_id] for entry in kept)) + return PhraseTable( + phrases=tuple(table[entry.phrase_id].model_copy(update={"default": counts[entry.phrase_id]}) for entry in kept) + ) diff --git a/src/sampletones_player/compression/encode.py b/src/sampletones_player/compression/encode.py index 589741739..f8620b982 100644 --- a/src/sampletones_player/compression/encode.py +++ b/src/sampletones_player/compression/encode.py @@ -1,13 +1,20 @@ +from collections import Counter from dataclasses import replace -from typing import Dict, Final, FrozenSet, Iterable, Sequence, Tuple +from typing import Dict, Final, FrozenSet, Iterable, List, Sequence, Tuple -from sampletones_player.compression.absent import is_absent from sampletones_player.compression.admit import admit_seeds -from sampletones_player.compression.budget import DEFAULT_SEARCH_BUDGET, SearchBudget +from sampletones_player.compression.budget import ( + DEFAULT_SEARCH_BUDGET, + SearchBudget, +) from sampletones_player.compression.compressed import CompressedPlanes from sampletones_player.compression.dictionary.phrase import Phrase from sampletones_player.compression.dictionary.prune import prune -from sampletones_player.compression.dictionary.table import PhraseTable, phrase_table +from sampletones_player.compression.dictionary.table import ( + PhraseTable, + phrase_table, +) +from sampletones_player.compression.entries import stream_entry from sampletones_player.compression.matches.cache import MatchCache from sampletones_player.compression.matches.index import PlaneIndex from sampletones_player.compression.options import CodecOptions @@ -15,6 +22,10 @@ from sampletones_player.compression.parse.song import parse_planes from sampletones_player.compression.planes.order import PlaneOrder from sampletones_player.compression.planes.song import SongPlanes +from sampletones_player.compression.planes.symbols import ( + pack_plane, + symbol_boundaries, +) from sampletones_player.compression.progress.monitor import CodecMonitor from sampletones_player.compression.progress.report import ( CodecReporter, @@ -24,7 +35,13 @@ from sampletones_player.compression.tokens.literal import LiteralToken from sampletones_player.compression.tokens.phrase import PhraseToken from sampletones_player.compression.tokens.types import TokenUnion -from sampletones_player.specification.compression import PHRASE_ID_ESCAPE, TokenTag +from sampletones_player.specification.compression import ( + DEFAULT_COUNT_FLAG, + NO_DEFAULT_COUNT, + PHRASE_ID_ESCAPE, + TokenTag, +) +from sampletones_player.specification.planes import PLANES from sampletones_shared.utils.progress import silent_reporter STREAM_START: Final[int] = 0 @@ -51,11 +68,13 @@ def emit(tokens: Sequence[TokenUnion]) -> bytes: case PhraseToken(): tag = TokenTag.TRANSPOSED_PHRASE if token.transpose else TokenTag.PHRASE named = min(token.phrase_id, PHRASE_ID_ESCAPE) - stream.append(tag | named) + stream.append(tag | (DEFAULT_COUNT_FLAG if token.default else 0) | named) if named == PHRASE_ID_ESCAPE: stream.append(token.phrase_id) - stream.append(token.ticks - 1) + if not token.default: + stream.append(token.ticks - 1) + if token.transpose: stream.append(token.transpose) @@ -90,6 +109,25 @@ def _savings( return savings +def _modal(played: Counter[int]) -> int: + """The count a phrase's tokens play it at most often, the shortest breaking a tie.""" + if not played: + return NO_DEFAULT_COUNT + + return min(played, key=lambda ticks: (-played[ticks], ticks)) + + +def _counts(parses: Iterable[Parse], phrases: int) -> Dict[int, int]: + """The count each phrase's tokens play it at most often, the shortest breaking a tie.""" + played: List[Counter[int]] = [Counter() for _ in range(phrases)] + for parse in parses: + for token in parse.tokens: + if isinstance(token, PhraseToken): + played[token.phrase_id][token.ticks] += 1 + + return {phrase_id: _modal(counter) for phrase_id, counter in enumerate(played)} + + def _settle( cache: MatchCache, table: PhraseTable, @@ -98,12 +136,19 @@ def _settle( monitor: CodecMonitor, baseline: Sequence[Parse], ) -> Tuple[PhraseTable, Tuple[Parse, ...]]: + """The table and its parses settled together, each round reading the other. + + A phrase's place, its id and the count its tokens leave unstated are all read off a parse + that was made under the table before it, so the table is rebuilt and the planes read again + until the two stand still. + """ parses = parse_planes(cache, table, options, boundaries, monitor) for _ in range(SETTLING_ROUNDS): pruned = prune( table, _references(parses, len(table)), _savings(parses, baseline, len(table)), + _counts(parses, len(table)), ) if pruned.phrases == table.phrases: break @@ -134,8 +179,8 @@ def encode_streams( phrases inside the opcodes that name them. Each plane is read against the shared dictionary on its own, so the planes may cover - different numbers of values. A plane playing only the value every plane starts at is absent: - it takes no stream, and the block states it with a sentinel the driver skips. + different numbers of values. A plane reaching the encoder empty is absent: it takes no + stream, and the block states it with a sentinel the driver skips. Args: planes: The planes, each at least one value long. @@ -157,7 +202,7 @@ def encode_streams( if len(boundaries) != len(planes): raise ValueError(f"a set of boundaries stands for each plane, and {len(boundaries)} stand for {len(planes)}") - present = [plane for plane, written in enumerate(planes) if not is_absent(written)] + present = [plane for plane, written in enumerate(planes) if written] cache = MatchCache(PlaneIndex.from_plane(planes[plane]) for plane in present) monitor = CodecMonitor(report) entries = tuple(boundaries[plane] | {STREAM_START} for plane in present) @@ -197,11 +242,16 @@ def encode_streams( baseline, ) written = iter(emit(parse.tokens) for parse in parses) - streams = tuple(b"" if is_absent(plane) else next(written) for plane in planes) + streams = tuple(next(written) if plane else b"" for plane in planes) monitor.reached(len(table), table.size + sum(len(stream) for stream in streams)) return table, streams +def _returned(boundaries: FrozenSet[int]) -> int: + """The tick a song comes round to, the song's first standing in where it plays once.""" + return min(boundaries, default=STREAM_START) + + def encode_planes( planes: SongPlanes, seeds: Sequence[Phrase], @@ -229,16 +279,37 @@ def encode_planes( OperationCanceled: If ``report`` withdraws the run. """ positions = [planes.positions(tick) for tick in boundaries] + entries = [frozenset(position[index] for position in positions) for index in range(len(planes.planes))] + packed = tuple( + (b"" if plane.idles(played) else pack_plane(played, plane.form, boundaries=entry)) + for plane, played, entry in zip(PLANES, planes.planes, entries, strict=True) + ) table, streams = encode_streams( - planes.planes, + packed, seeds, options=options, - boundaries=[frozenset(position[plane] for position in positions) for plane in range(len(planes.planes))], + boundaries=[ + symbol_boundaries(played, plane.form, boundaries=entry) + for plane, played, entry in zip(PLANES, planes.planes, entries, strict=True) + ], budget=budget, report=report, ) + returned = [ + symbol_boundaries(played, plane.form, boundaries=frozenset({position})) + for plane, played, position in zip( + PLANES, + planes.planes, + planes.positions(_returned(boundaries)), + strict=True, + ) + ] return CompressedPlanes( phrases=table, streams=PlaneOrder.across(streams), ticks=planes.ticks, + loop_entries=tuple( + (stream_entry(stream, next(iter(entry), STREAM_START), table) if stream else None) + for stream, entry in zip(streams, returned, strict=True) + ), ) diff --git a/src/sampletones_player/compression/entries.py b/src/sampletones_player/compression/entries.py index 5cf25d643..ab3ffe58b 100644 --- a/src/sampletones_player/compression/entries.py +++ b/src/sampletones_player/compression/entries.py @@ -1,7 +1,12 @@ +from sampletones_player.compression.dictionary.table import PhraseTable from sampletones_player.compression.tokens.span import token_span -def stream_entry(stream: bytes, position: int) -> int: +def stream_entry( + stream: bytes, + position: int, + table: PhraseTable, +) -> int: """The byte of ``stream`` the token covering the value at ``position`` begins at. A song that repeats re-enters its streams partway through, and what the driver needs to @@ -11,6 +16,7 @@ def stream_entry(stream: bytes, position: int) -> int: Args: stream: The plane's token stream. position: The value the stream is re-entered at. + table: The dictionary the tokens name. Returns: int: The byte the token covering ``position`` begins at, counted from the stream's own @@ -22,7 +28,7 @@ def stream_entry(stream: bytes, position: int) -> int: offset = 0 reached = 0 while reached < position: - span = token_span(stream, offset) + span = token_span(stream, offset, table) reached += span.ticks offset += span.size diff --git a/src/sampletones_player/compression/matches/cache.py b/src/sampletones_player/compression/matches/cache.py index e844a6d3b..1883000a0 100644 --- a/src/sampletones_player/compression/matches/cache.py +++ b/src/sampletones_player/compression/matches/cache.py @@ -5,7 +5,11 @@ from sampletones_player.compression.matches.index import PlaneIndex from sampletones_player.compression.matches.played import played_ticks from sampletones_player.compression.matches.reading import PhraseReading -from sampletones_player.compression.matches.shift import NO_SHIFT, asked_shift, translation +from sampletones_player.compression.matches.shift import ( + NO_SHIFT, + asked_shift, + translation, +) from sampletones_player.specification.compression import MAX_PHRASE_TICKS KEY_LENGTH: Final[int] = 2 diff --git a/src/sampletones_player/compression/matches/match.py b/src/sampletones_player/compression/matches/match.py deleted file mode 100644 index ebfd4c4c9..000000000 --- a/src/sampletones_player/compression/matches/match.py +++ /dev/null @@ -1,15 +0,0 @@ -from typing import NamedTuple - - -class PhraseMatch(NamedTuple): - """A phrase the plane plays from a tick, and the terms it plays it on. - - Attributes: - phrase_id: Position the phrase takes in the table. - ticks: The ticks the plane plays of it, its final value held past its end. - transpose: The shift every byte of it is played at. - """ - - phrase_id: int - ticks: int - transpose: int diff --git a/src/sampletones_player/compression/matches/matcher.py b/src/sampletones_player/compression/matches/matcher.py index 89e772271..c6ec842bb 100644 --- a/src/sampletones_player/compression/matches/matcher.py +++ b/src/sampletones_player/compression/matches/matcher.py @@ -1,13 +1,30 @@ from itertools import chain -from typing import Dict, Iterator, List, Sequence, Tuple +from typing import Dict, Iterator, List, NamedTuple, Sequence, Tuple from sampletones_player.compression.dictionary.table import PhraseTable -from sampletones_player.compression.matches.cache import KEY_LENGTH, MIN_PHRASE_TICKS, MatchCache +from sampletones_player.compression.matches.cache import ( + KEY_LENGTH, + MIN_PHRASE_TICKS, + MatchCache, +) from sampletones_player.compression.matches.index import PlaneIndex -from sampletones_player.compression.matches.match import PhraseMatch from sampletones_player.specification.binary import BYTE_VALUES +class PhraseMatch(NamedTuple): + """A phrase the plane plays from a tick, and the terms it plays it on. + + Attributes: + phrase_id: Position the phrase takes in the table. + ticks: The ticks the plane plays of it, its final value held past its end. + transpose: The shift every byte of it is played at. + """ + + phrase_id: int + ticks: int + transpose: int + + class PhraseMatcher: """Answers which phrases one plane plays at a tick, and for how many ticks. @@ -31,6 +48,7 @@ def __init__( cache: MatchCache, ) -> None: self._index: PlaneIndex = cache.index(plane) + self._table: PhraseTable = table self._origins: Tuple[int, ...] = tuple(phrase.body[0] for phrase in table.phrases) self._played: Tuple[Sequence[int], ...] = tuple(cache.reading(plane, phrase).ticks for phrase in table.phrases) keyed: Dict[bytes, List[int]] = {} @@ -46,6 +64,11 @@ def __init__( self._keyed: Dict[bytes, Tuple[int, ...]] = {key: tuple(ids) for key, ids in keyed.items()} self._short: Tuple[int, ...] = tuple(short) + @property + def table(self) -> PhraseTable: + """The dictionary the matcher names phrases from.""" + return self._table + @property def index(self) -> PlaneIndex: """The plane the matcher answers for, and the readings of it matching is decided against.""" diff --git a/src/sampletones_player/compression/parse/plane.py b/src/sampletones_player/compression/parse/plane.py index 53ce7146e..2f7a7b5a4 100644 --- a/src/sampletones_player/compression/parse/plane.py +++ b/src/sampletones_player/compression/parse/plane.py @@ -19,6 +19,7 @@ MAX_HOLD_TICKS, MAX_LITERAL_BYTES, MAX_PHRASE_TICKS, + NO_DEFAULT_COUNT, ) @@ -33,7 +34,12 @@ def _relax_literal( start = window.cheapest(position, earliest) cost = shortest.costs[start] + literal_size(position - start) if shortest.improves(position, cost): - shortest.relax(start, position, cost, LiteralToken(values=plane[start:position])) + shortest.relax( + start, + position, + cost, + LiteralToken(values=plane[start:position]), + ) def _relax_forward( @@ -69,9 +75,27 @@ def _relax_forward( shortest.relax( position, position + ticks, - cost + phrase_size(phrase_id, transpose), - PhraseToken(phrase_id=phrase_id, ticks=ticks, transpose=transpose), + cost + phrase_size(phrase_id, transpose, default=False), + PhraseToken( + phrase_id=phrase_id, + ticks=ticks, + transpose=transpose, + default=False, + ), ) + carried = matcher.table[phrase_id].default + if NO_DEFAULT_COUNT < carried <= ticks: + shortest.relax( + position, + position + carried, + cost + phrase_size(phrase_id, transpose, default=True), + PhraseToken( + phrase_id=phrase_id, + ticks=carried, + transpose=transpose, + default=True, + ), + ) def parse_plane( diff --git a/src/sampletones_player/compression/planes/channel.py b/src/sampletones_player/compression/planes/channel.py deleted file mode 100644 index db5fc91ad..000000000 --- a/src/sampletones_player/compression/planes/channel.py +++ /dev/null @@ -1,94 +0,0 @@ -from __future__ import annotations - -from typing import Tuple - -from pydantic import BaseModel, ConfigDict, model_validator - -from sampletones_player.compression.planes.flags import flagged_ticks - - -class ChannelPlanes(BaseModel): - """One channel's ticks separated into the byte series it writes. - - A channel writes two things each tick: how it sounds and what it sounds. Read tick by tick - those two braid together, and each turns over at its own pace — a volume envelope decays - while a pitch holds, a pitch walks while the timbre stays put. Kept apart, each is a series - that repeats and rests on its own terms, which is the form the codec reads them in. - - Attributes: - control: The timbre byte each tick writes, volume riding in it where a channel has one. - value: The pitch each tick sounds, as an index into the pitch table, or the noise - channel's period byte. - """ - - model_config = ConfigDict(extra="forbid", frozen=True) - - control: bytes - value: bytes - - @model_validator(mode="after") - def _validate_every_plane_covers_the_same_ticks(self) -> ChannelPlanes: - if len(self.control) != len(self.value): - raise ValueError( - f"a channel's control and value cover the same ticks, and these cover " - f"{len(self.control)} and {len(self.value)}" - ) - - if not self.control: - raise ValueError("a channel's planes cover at least one tick") - - return self - - @property - def ticks(self) -> int: - """The ticks the channel's planes cover.""" - return len(self.control) - - @property - def ordered(self) -> Tuple[bytes, ...]: - """The channel's planes, in the order the song block writes them.""" - return (self.control, self.value) - - -class TonePlanes(ChannelPlanes): - """A tone channel's ticks, the divider each one sounds at named in two parts. - - A tone channel reaches its divider through the pitch table, and a frame may stand away from - the note it names. The value plane holds the note, which is what lets a phrase be transposed - by adding to it, and the bend plane holds how far the frame stands from it — so the divider - the hardware takes is the sum, and each half repeats on its own terms. - - The bend plane is read on the ticks the value plane flags and on those alone: a tick left - unflagged sounds its note's own divider. A channel that never bends therefore holds an empty - bend plane, and one that bends holds a value only where a note does. - - Attributes: - bend: The divider steps each flagged tick stands away from its note, held as signed bytes - in the order the flagged ticks come. - """ - - bend: bytes - - @model_validator(mode="after") - def _validate_the_bend_covers_the_flagged_ticks(self) -> TonePlanes: - flagged = flagged_ticks(self.value) - if len(self.bend) != flagged: - raise ValueError(f"a bend plane holds a value per flagged tick, {flagged}, and this holds {len(self.bend)}") - - return self - - def bend_position(self, tick: int) -> int: - """Where in the bend plane the song stands once ``tick`` ticks have played. - - Args: - tick: The ticks played. - - Returns: - int: The bend values those ticks read. - """ - return flagged_ticks(self.value[:tick]) - - @property - def ordered(self) -> Tuple[bytes, ...]: - """The channel's planes, in the order the song block writes them.""" - return (self.control, self.value, self.bend) diff --git a/src/sampletones_player/compression/planes/flags.py b/src/sampletones_player/compression/planes/flags.py index 65d858cbb..92a641d83 100644 --- a/src/sampletones_player/compression/planes/flags.py +++ b/src/sampletones_player/compression/planes/flags.py @@ -1,6 +1,9 @@ from typing import List, Sequence, Tuple -from sampletones_player.specification.compression import BEND_FLAG, PITCH_INDEX_MASK +from sampletones_player.specification.compression import ( + BEND_FLAG, + PITCH_INDEX_MASK, +) def is_flagged(value: int) -> bool: diff --git a/src/sampletones_player/compression/planes/order.py b/src/sampletones_player/compression/planes/order.py index 2291484af..35133463a 100644 --- a/src/sampletones_player/compression/planes/order.py +++ b/src/sampletones_player/compression/planes/order.py @@ -2,7 +2,7 @@ from typing import Iterable, NamedTuple, Tuple -from sampletones_player.specification.compression import PLANE_COUNT +from sampletones_player.specification.planes import PLANE_COUNT, PLANE_NAMES class PlaneOrder(NamedTuple): @@ -12,6 +12,10 @@ class PlaneOrder(NamedTuple): shape: the values each plane plays tick by tick, and the tokens those values are written as. Naming them is what lets either be carried whole and read back by the channel it belongs to. + The fields spell out what ``specification.planes.PLANES`` states, so that a reader reaches a + plane by its own name and a type checker knows which names there are. A test holds the two + in step. + Attributes: pulse1_control: The first pulse channel's timbre and volume. pulse1_value: The first pulse channel's pitch. @@ -19,8 +23,7 @@ class PlaneOrder(NamedTuple): pulse2_control: The second pulse channel's timbre and volume. pulse2_value: The second pulse channel's pitch. pulse2_bend: The second pulse channel's divider offset. - triangle_control: The triangle channel's linear counter. - triangle_value: The triangle channel's pitch. + triangle_value: The triangle channel's pitch, the index above the table naming a rest. triangle_bend: The triangle channel's divider offset. noise_control: The noise channel's timbre and volume. noise_value: The noise channel's period. @@ -32,7 +35,6 @@ class PlaneOrder(NamedTuple): pulse2_control: bytes pulse2_value: bytes pulse2_bend: bytes - triangle_control: bytes triangle_value: bytes triangle_bend: bytes noise_control: bytes @@ -41,10 +43,13 @@ class PlaneOrder(NamedTuple): @classmethod def names(cls) -> Tuple[str, ...]: """The planes' names, in the order the song block writes them.""" - return cls._fields + return PLANE_NAMES @classmethod - def across(cls, planes: Iterable[bytes]) -> PlaneOrder: + def across( + cls, + planes: Iterable[bytes], + ) -> PlaneOrder: """Gathers a song's planes under the names the song block writes them by. Args: diff --git a/src/sampletones_player/compression/planes/rebuild.py b/src/sampletones_player/compression/planes/rebuild.py index 057174453..9d78c1680 100644 --- a/src/sampletones_player/compression/planes/rebuild.py +++ b/src/sampletones_player/compression/planes/rebuild.py @@ -1,7 +1,7 @@ -from typing import Iterator, Tuple +from typing import Final, Iterator, List, Tuple +from sampletones_core.constants.enums import ChannelName from sampletones_player.compression.pitch import PitchTable -from sampletones_player.compression.planes.channel import ChannelPlanes, TonePlanes from sampletones_player.compression.planes.flags import is_flagged, pitch_index from sampletones_player.compression.planes.song import SongPlanes from sampletones_player.registers.noise import NoiseRegisters @@ -9,49 +9,85 @@ from sampletones_player.registers.streams import ChannelStreams from sampletones_player.registers.triangle import TriangleRegisters from sampletones_player.specification.binary import signed_byte +from sampletones_player.specification.planes import SILENT_PITCH_INDEX from sampletones_player.specification.registers import ( MAX_REGISTER_VALUE, TIMER_HIGH_SHIFT, + TRIANGLE_COUNTER_CONTROL, + TRIANGLE_SILENT_RELOAD, + TRIANGLE_SOUNDING_RELOAD, ) +FIRST_PITCH: Final[int] = 0 + def tone_dividers( - planes: TonePlanes, + value: bytes, + bend: bytes, timers: Tuple[int, ...], ) -> Tuple[int, ...]: """The divider each tick of a tone channel reaches, its note and its bend together. This is the reading the driver performs between the pitch table and the timer registers, stated where it is testable: a flagged tick takes the bend plane's next value, and every - other tick sounds its pitch's own divider. + other tick sounds its pitch's own divider. A tick naming the index that stands for silence + holds the divider the channel last sounded, and the lowest pitch's before it sounds at all. Args: - planes: The channel's planes. + value: The pitch each tick names, under the flag saying whether it bends. + bend: The steps each flagged tick stands from its pitch's own divider. timers: The divider each pitch sounds at, in pitch order. Returns: Tuple[int, ...]: One divider per tick. """ - bends = iter(planes.bend) - return tuple( - timers[pitch_index(value)] + (signed_byte(next(bends)) if is_flagged(value) else 0) for value in planes.value - ) + bends = iter(bend) + dividers: List[int] = [] + held = timers[FIRST_PITCH] + for named in value: + if named != SILENT_PITCH_INDEX: + held = timers[pitch_index(named)] + (signed_byte(next(bends)) if is_flagged(named) else 0) + + dividers.append(held) + + return tuple(dividers) + + +def held_pitches(value: bytes) -> Tuple[int, ...]: + """The pitch index each tick sounds at, a rest holding the one the channel last sounded. + + Args: + value: The pitch each tick names, the index above the table naming a rest. + + Returns: + Tuple[int, ...]: One index per tick. + """ + indices: List[int] = [] + held = FIRST_PITCH + for named in value: + if named != SILENT_PITCH_INDEX: + held = pitch_index(named) + + indices.append(held) + + return tuple(indices) def _sounded( - planes: TonePlanes, + planes: Tuple[bytes, ...], pitches: PitchTable, ) -> Iterator[Tuple[int, int, int]]: """Each tick's control byte, the divider its registers carry, and the pitch it is counted from.""" + control, value, bend = planes yield from zip( - planes.control, - tone_dividers(planes, pitches.timers), - (pitches.pitch(pitch_index(value)) for value in planes.value), + control, + tone_dividers(value, bend, pitches.timers), + (pitches.pitch(pitch_index(named)) for named in value), ) def _pulse_registers( - planes: TonePlanes, + planes: Tuple[bytes, ...], pitches: PitchTable, ) -> Tuple[PulseRegisters, ...]: return tuple( @@ -66,27 +102,37 @@ def _pulse_registers( def _triangle_registers( - planes: TonePlanes, + planes: Tuple[bytes, ...], pitches: PitchTable, ) -> Tuple[TriangleRegisters, ...]: + """The triangle's ticks, its linear counter read from the pitch its value plane names. + + An index above every pitch the table holds names a rest, which silences the counter and + leaves the divider where the channel last sounded. + """ + value, bend = planes + dividers = tone_dividers(value, bend, pitches.timers) + anchors = held_pitches(value) return tuple( TriangleRegisters( - linear_counter=control, + linear_counter=TRIANGLE_COUNTER_CONTROL + | (TRIANGLE_SILENT_RELOAD if named == SILENT_PITCH_INDEX else TRIANGLE_SOUNDING_RELOAD), timer_low=timer & MAX_REGISTER_VALUE, timer_high=timer >> TIMER_HIGH_SHIFT, - anchor=anchor, + anchor=pitches.pitch(index), ) - for control, timer, anchor in _sounded(planes, pitches) + for named, timer, index in zip(value, dividers, anchors, strict=True) ) -def _noise_registers(planes: ChannelPlanes) -> Tuple[NoiseRegisters, ...]: +def _noise_registers(planes: Tuple[bytes, ...]) -> Tuple[NoiseRegisters, ...]: + control, value = planes return tuple( NoiseRegisters( - control=control, + control=timbre, period=period, ) - for control, period in zip(planes.control, planes.value) + for timbre, period in zip(control, value) ) @@ -97,15 +143,15 @@ def streams_from_planes( """Rebuilds a song's four streams from the planes they were separated into. Args: - planes: The planes under the channel each belongs to. + planes: Every plane, in the order the song block writes them. pitches: The timer each pitch sounds at. Returns: ChannelStreams: The per-tick register values every channel plays. """ return ChannelStreams( - pulse1=_pulse_registers(planes.pulse1, pitches), - pulse2=_pulse_registers(planes.pulse2, pitches), - triangle=_triangle_registers(planes.triangle, pitches), - noise=_noise_registers(planes.noise), + pulse1=_pulse_registers(planes.of(ChannelName.PULSE1), pitches), + pulse2=_pulse_registers(planes.of(ChannelName.PULSE2), pitches), + triangle=_triangle_registers(planes.of(ChannelName.TRIANGLE), pitches), + noise=_noise_registers(planes.of(ChannelName.NOISE)), ) diff --git a/src/sampletones_player/compression/planes/separate.py b/src/sampletones_player/compression/planes/separate.py index e2ca3497a..0563e9c06 100644 --- a/src/sampletones_player/compression/planes/separate.py +++ b/src/sampletones_player/compression/planes/separate.py @@ -1,14 +1,20 @@ -from typing import Final, List, Sequence +from typing import Final, List, Sequence, Tuple from sampletones_core.constants.enums import TONE_CHANNELS, ChannelName from sampletones_player.compression.pitch import PitchTable -from sampletones_player.compression.planes.channel import ChannelPlanes, TonePlanes -from sampletones_player.compression.planes.flags import flagged_value, note_flags +from sampletones_player.compression.planes.flags import ( + flagged_value, + is_flagged, + note_flags, +) +from sampletones_player.compression.planes.order import PlaneOrder from sampletones_player.compression.planes.song import SongPlanes from sampletones_player.registers.base import ChannelRegisters from sampletones_player.registers.streams import ChannelStreams from sampletones_player.registers.tone import ToneRegisters from sampletones_player.specification.binary import unsigned_byte +from sampletones_player.specification.planes import SILENT_PITCH_INDEX +from sampletones_player.specification.registers import TRIANGLE_SOUNDING_RELOAD CONTROL_VALUE_INDEX: Final[int] = 0 FIRST_VALUE_INDEX: Final[int] = 1 @@ -29,29 +35,56 @@ def _tone_ticks(registers: Sequence[ChannelRegisters]) -> List[ToneRegisters]: def _tone_planes( registers: Sequence[ChannelRegisters], pitches: PitchTable, -) -> TonePlanes: +) -> Tuple[bytes, ...]: ticks = _tone_ticks(registers) indices = [pitches.index(tick.anchor) for tick in ticks] offsets = [tick.divider - pitches.timers[index] for tick, index in zip(ticks, indices)] flags = note_flags(indices, offsets) - return TonePlanes( - control=bytes(tick.values[CONTROL_VALUE_INDEX] for tick in ticks), - value=bytes(flagged_value(index, flag) for index, flag in zip(indices, flags)), - bend=bytes(unsigned_byte(offset) for offset, flag in zip(offsets, flags) if flag), + return ( + bytes(tick.values[CONTROL_VALUE_INDEX] for tick in ticks), + bytes(flagged_value(index, flag) for index, flag in zip(indices, flags)), + bytes(unsigned_byte(offset) for offset, flag in zip(offsets, flags) if flag), ) -def _noise_planes(registers: Sequence[ChannelRegisters]) -> ChannelPlanes: - control = bytes(tick.values[CONTROL_VALUE_INDEX] for tick in registers) - value = bytes(tick.values[FIRST_VALUE_INDEX] for tick in registers) - return ChannelPlanes(control=control, value=value) +def _noise_planes(registers: Sequence[ChannelRegisters]) -> Tuple[bytes, ...]: + return ( + bytes(tick.values[CONTROL_VALUE_INDEX] for tick in registers), + bytes(tick.values[FIRST_VALUE_INDEX] for tick in registers), + ) + + +def _triangle_planes( + registers: Sequence[ChannelRegisters], + pitches: PitchTable, +) -> Tuple[bytes, ...]: + """The triangle's planes, its silence named in the pitch its value plane carries. + + The channel sounds at one level, so a tick states whether it sounds through the index it + names: the index standing above every pitch the table holds silences the linear counter, and + the divider stays where the channel last sounded. A resting tick names no pitch, so it bends + nowhere and its bend plane holds nothing for it. + """ + control, value, bend = _tone_planes(registers, pitches) + offsets = iter(bend) + named = bytearray() + bent = bytearray() + for timbre, pitch in zip(control, value, strict=True): + sounding = bool(timbre & TRIANGLE_SOUNDING_RELOAD) + flagged = is_flagged(pitch) + offset = next(offsets) if flagged else None + named.append(pitch if sounding else SILENT_PITCH_INDEX) + if sounding and offset is not None: + bent.append(offset) + + return (bytes(named), bytes(bent)) def channel_planes( channel: ChannelName, registers: Sequence[ChannelRegisters], pitches: PitchTable, -) -> ChannelPlanes: +) -> Tuple[bytes, ...]: """Separates one channel's ticks into the planes the codec reads. A tone channel's value plane names the pitch each divider is counted from, and its bend plane @@ -64,17 +97,20 @@ def channel_planes( pitches: The timer each pitch sounds at. Returns: - ChannelPlanes: The channel's own planes. + Tuple[bytes, ...]: The channel's own planes, in the order the song block writes them. Raises: TypeError: If a tone channel's ticks hold registers another channel writes. ValueError: If a tone channel's divider lies further from the pitch it is counted from than a signed byte states. """ - if channel in TONE_CHANNELS: - return _tone_planes(registers, pitches) - - return _noise_planes(registers) + match channel: + case ChannelName.TRIANGLE: + return _triangle_planes(registers, pitches) + case _ if channel in TONE_CHANNELS: + return _tone_planes(registers, pitches) + case _: + return _noise_planes(registers) def planes_from_streams( @@ -97,10 +133,10 @@ def planes_from_streams( ValueError: If a tone channel's divider lies further from the pitch it is counted from than a signed byte states. """ - pulse1, pulse2, triangle, noise = streams.padded return SongPlanes( - pulse1=_tone_planes(pulse1, pitches), - pulse2=_tone_planes(pulse2, pitches), - triangle=_tone_planes(triangle, pitches), - noise=_noise_planes(noise), + planes=PlaneOrder.across( + plane + for channel, registers in zip(ChannelName.items(), streams.padded, strict=True) + for plane in channel_planes(channel, registers, pitches) + ) ) diff --git a/src/sampletones_player/compression/planes/song.py b/src/sampletones_player/compression/planes/song.py index 3e7ba5ecf..a4754a55c 100644 --- a/src/sampletones_player/compression/planes/song.py +++ b/src/sampletones_player/compression/planes/song.py @@ -1,99 +1,76 @@ from __future__ import annotations -from typing import Tuple +from typing import Final, Tuple from pydantic import BaseModel, ConfigDict, model_validator -from sampletones_player.compression.planes.channel import ChannelPlanes, TonePlanes +from sampletones_core.constants.enums import ChannelName +from sampletones_player.compression.planes.flags import flagged_ticks from sampletones_player.compression.planes.order import PlaneOrder +from sampletones_player.specification.planes import ( + PLANES, + PlaneRole, + channel_indices, + plane_index, +) + +FIRST_PLANE: Final[int] = 0 class SongPlanes(BaseModel): - """Every channel of a song separated into planes, the whole of what the codec compresses. + """Every plane of a song, in the order the song block writes them. + + A channel writes what it sounds and how it sounds it as separate byte series. Read tick by + tick those braid together, and each turns over at its own pace — a volume envelope decays + while a pitch holds, a pitch walks while the timbre stays put. Kept apart, each is a series + that repeats and rests on its own terms, which is the form the codec reads them in. + + A bend plane holds a value on the ticks its channel's value plane flags and on those alone, + so it covers fewer ticks than the song lasts while every other plane covers them all. Attributes: - pulse1: The first pulse channel's planes. - pulse2: The second pulse channel's planes. - triangle: The triangle channel's planes. - noise: The noise channel's planes. + planes: The byte series each plane plays, under the name the song block writes it by. """ model_config = ConfigDict(extra="forbid", frozen=True) - pulse1: TonePlanes - pulse2: TonePlanes - triangle: TonePlanes - noise: ChannelPlanes - - @classmethod - def from_order(cls, planes: PlaneOrder) -> SongPlanes: - """Gathers a song block's planes back into the four channels that write them. - - Args: - planes: The planes, in the order the song block writes them. - - Returns: - SongPlanes: The planes under the channel each belongs to. - """ - return cls( - pulse1=TonePlanes( - control=planes.pulse1_control, - value=planes.pulse1_value, - bend=planes.pulse1_bend, - ), - pulse2=TonePlanes( - control=planes.pulse2_control, - value=planes.pulse2_value, - bend=planes.pulse2_bend, - ), - triangle=TonePlanes( - control=planes.triangle_control, - value=planes.triangle_value, - bend=planes.triangle_bend, - ), - noise=ChannelPlanes( - control=planes.noise_control, - value=planes.noise_value, - ), - ) + planes: PlaneOrder @model_validator(mode="after") - def _validate_every_channel_reaches_the_same_tick(self) -> SongPlanes: - lengths = {channels.ticks for channels in self.ordered} - if len(lengths) > 1: - raise ValueError(f"a song's channels cover the same ticks, and these cover {sorted(lengths)}") + def _validate_every_plane_covers_the_song(self) -> SongPlanes: + if not self.planes[FIRST_PLANE]: + raise ValueError("a song's planes cover at least one tick") - return self + for plane, played in zip(PLANES, self.planes, strict=True): + reach = self._flagged(plane.channel) if plane.spans_flagged_ticks else self.ticks + if len(played) != reach: + raise ValueError( + f"the {plane.name} plane covers {reach} of the song's values, and this covers {len(played)}" + ) - @property - def ordered(self) -> Tuple[ChannelPlanes, ...]: - """The four channels in the order the generator names run.""" - return (self.pulse1, self.pulse2, self.triangle, self.noise) + return self - @property - def planes(self) -> PlaneOrder: - """Every plane in the order the song block writes them.""" - return PlaneOrder( - pulse1_control=self.pulse1.control, - pulse1_value=self.pulse1.value, - pulse1_bend=self.pulse1.bend, - pulse2_control=self.pulse2.control, - pulse2_value=self.pulse2.value, - pulse2_bend=self.pulse2.bend, - triangle_control=self.triangle.control, - triangle_value=self.triangle.value, - triangle_bend=self.triangle.bend, - noise_control=self.noise.control, - noise_value=self.noise.value, - ) + def _flagged(self, channel: ChannelName) -> int: + return flagged_ticks(self.planes[plane_index(channel, PlaneRole.VALUE)]) @property def ticks(self) -> int: """The ticks the song lasts.""" - return self.pulse1.ticks + return len(self.planes[FIRST_PLANE]) + + def of(self, channel: ChannelName) -> Tuple[bytes, ...]: + """The planes ``channel`` writes, in the order the song block writes them. + + Args: + channel: The channel to read. + + Returns: + Tuple[bytes, ...]: That channel's own planes. + """ + return tuple(self.planes[index] for index in channel_indices(channel)) def positions(self, tick: int) -> Tuple[int, ...]: - """Where each plane stands once ``tick`` ticks have played, in the order the block writes them. + """Where each plane stands once ``tick`` ticks have played, in block order. Every plane reads one value a tick but a bend plane, which reads one on each tick its channel flags, so a song returning to a tick re-enters each plane at a position of its own. @@ -104,16 +81,11 @@ def positions(self, tick: int) -> Tuple[int, ...]: Returns: Tuple[int, ...]: One position per plane. """ - return ( - tick, - tick, - self.pulse1.bend_position(tick), - tick, - tick, - self.pulse2.bend_position(tick), - tick, - tick, - self.triangle.bend_position(tick), - tick, - tick, + return tuple( + ( + flagged_ticks(self.planes[plane_index(plane.channel, PlaneRole.VALUE)][:tick]) + if plane.spans_flagged_ticks + else tick + ) + for plane in PLANES ) diff --git a/src/sampletones_player/compression/planes/symbols.py b/src/sampletones_player/compression/planes/symbols.py new file mode 100644 index 000000000..6c1093121 --- /dev/null +++ b/src/sampletones_player/compression/planes/symbols.py @@ -0,0 +1,104 @@ +from typing import FrozenSet, List, Tuple + +from sampletones_player.specification.planes import SINGLE_TICK, PlaneForm + + +def pack_plane( + played: bytes, + form: PlaneForm, + *, + boundaries: FrozenSet[int], +) -> bytes: + """The symbols a plane plays, each a value and the ticks it repeats for. + + A run of one value becomes one symbol for as many ticks as the form's count reaches, and a + longer run becomes as many symbols as that takes. A tick the song re-enters on begins a + symbol of its own, so the plane stands at a symbol's first byte wherever a token may start. + + Args: + played: The values the plane plays. + form: How the plane's byte divides. + boundaries: The ticks a symbol begins on, beyond the plane's first. + + Returns: + bytes: The symbols, in the order they are played. + + Raises: + ValueError: If a value the plane plays carries fixed bits other than the form's. + """ + symbols = bytearray() + for start, length in _runs(played, boundaries=boundaries): + held = length + while held: + repeats = min(held, form.repeats) + symbols.append(form.symbol(played[start], repeats)) + held -= repeats + + return bytes(symbols) + + +def unpack_plane( + symbols: bytes, + form: PlaneForm, +) -> bytes: + """The values a run of symbols plays, one per tick. + + Args: + symbols: The symbols the plane is written as. + form: How the plane's byte divides. + + Returns: + bytes: The values, one per tick. + """ + values = bytearray() + for symbol in symbols: + values.extend((form.value(symbol),) * form.repeated(symbol)) + + return bytes(values) + + +def symbol_boundaries( + played: bytes, + form: PlaneForm, + *, + boundaries: FrozenSet[int], +) -> FrozenSet[int]: + """Where the ticks a token starts on stand once the plane is packed. + + A packed plane is read a symbol at a time, so the parse counts symbols where it counted + ticks and a boundary reaches it as the symbol the tick begins. + + Args: + played: The values the plane plays. + form: How the plane's byte divides. + boundaries: The ticks a token starts on, beyond the plane's first. + + Returns: + FrozenSet[int]: The symbols those ticks begin. + """ + positions = {} + symbol = 0 + for start, length in _runs(played, boundaries=boundaries): + positions[start] = symbol + symbol += -(-length // form.repeats) + + return frozenset(positions[tick] for tick in boundaries if tick in positions) + + +def _runs( + played: bytes, + *, + boundaries: FrozenSet[int], +) -> Tuple[Tuple[int, int], ...]: + """Where each run of one value begins and how many ticks it lasts, cut at every boundary.""" + runs: List[Tuple[int, int]] = [] + start = 0 + for tick in range(SINGLE_TICK, len(played)): + if played[tick] != played[start] or tick in boundaries: + runs.append((start, tick - start)) + start = tick + + if played: + runs.append((start, len(played) - start)) + + return tuple(runs) diff --git a/src/sampletones_player/compression/progress/monitor.py b/src/sampletones_player/compression/progress/monitor.py index c38c31864..16bfcd6de 100644 --- a/src/sampletones_player/compression/progress/monitor.py +++ b/src/sampletones_player/compression/progress/monitor.py @@ -1,6 +1,9 @@ from typing import Final -from sampletones_player.compression.progress.report import CodecProgress, CodecReporter +from sampletones_player.compression.progress.report import ( + CodecProgress, + CodecReporter, +) from sampletones_shared.exceptions import OperationCanceled NOTHING_FOUND: Final[int] = 0 diff --git a/src/sampletones_player/compression/scheme.py b/src/sampletones_player/compression/scheme.py index 28d3814f0..7430b321a 100644 --- a/src/sampletones_player/compression/scheme.py +++ b/src/sampletones_player/compression/scheme.py @@ -27,7 +27,12 @@ def options(self) -> CodecOptions: """ match self: case CompressionScheme.NONE: - return CodecOptions(holds=False, phrases=False, transposition=False, search=False) + return CodecOptions( + holds=False, + phrases=False, + transposition=False, + search=False, + ) case CompressionScheme.RUNS: return CodecOptions(holds=True, phrases=False, transposition=False, search=False) case CompressionScheme.INSTRUMENTS: diff --git a/src/sampletones_player/compression/search.py b/src/sampletones_player/compression/search.py index 300208343..019d4b409 100644 --- a/src/sampletones_player/compression/search.py +++ b/src/sampletones_player/compression/search.py @@ -1,13 +1,32 @@ -from typing import Callable, Dict, Final, FrozenSet, Iterator, List, NamedTuple, Sequence, Tuple +from typing import ( + Callable, + Dict, + Final, + FrozenSet, + Iterator, + List, + NamedTuple, + Sequence, + Tuple, +) from sampletones_player.compression.budget import SearchBudget, shares -from sampletones_player.compression.dictionary.phrase import Phrase, phrase_entry_size -from sampletones_player.compression.dictionary.table import PhraseTable, phrase_table +from sampletones_player.compression.dictionary.phrase import ( + Phrase, + phrase_entry_size, +) +from sampletones_player.compression.dictionary.table import ( + PhraseTable, + phrase_table, +) from sampletones_player.compression.matches.cache import MatchCache from sampletones_player.compression.matches.index import PlaneIndex from sampletones_player.compression.options import CodecOptions from sampletones_player.compression.parse.result import Parse -from sampletones_player.compression.parse.song import parse_planes, parse_planes_offered +from sampletones_player.compression.parse.song import ( + parse_planes, + parse_planes_offered, +) from sampletones_player.compression.progress.monitor import CodecMonitor from sampletones_player.compression.tokens.literal import LiteralToken from sampletones_player.compression.tokens.sizes import phrase_size @@ -130,8 +149,8 @@ def _gain( costs = parses[occurrence.plane].costs parsed += costs[occurrence.position + length] - costs[occurrence.position] - stated = phrase_size(phrase_id, UNSHIFTED_OCCURRENCE_TRANSPOSE) - shifted = phrase_size(phrase_id, SHIFTED_OCCURRENCE_TRANSPOSE) + stated = phrase_size(phrase_id, UNSHIFTED_OCCURRENCE_TRANSPOSE, default=False) + shifted = phrase_size(phrase_id, SHIFTED_OCCURRENCE_TRANSPOSE, default=False) return parsed - stated - shifted * (len(occurrences) - 1) diff --git a/src/sampletones_player/compression/seeds.py b/src/sampletones_player/compression/seeds.py index 665a64f0f..466271b51 100644 --- a/src/sampletones_player/compression/seeds.py +++ b/src/sampletones_player/compression/seeds.py @@ -1,4 +1,4 @@ -from typing import AbstractSet, List, Tuple +from typing import AbstractSet, Final, FrozenSet, List, Tuple from sampletones_core.constants.enums import ChannelName from sampletones_core.exporters.maps import CHANNEL_TO_EXPORTER_MAP @@ -8,10 +8,14 @@ from sampletones_player.compression.dictionary.phrase import Phrase from sampletones_player.compression.pitch import PitchTable from sampletones_player.compression.planes.separate import channel_planes +from sampletones_player.compression.planes.symbols import pack_plane from sampletones_player.registers.channel import channel_registers from sampletones_player.specification.compression import MAX_PHRASE_LENGTH +from sampletones_player.specification.planes import PLANES, channel_indices from sampletones_shared.music import Tuning +NO_BOUNDARIES: Final[FrozenSet[int]] = frozenset() + def phrases_from_project( project: Project, @@ -26,7 +30,8 @@ def phrases_from_project( row asks for. A plane holding one value throughout offers the dictionary nothing a hold covers more - cheaply, so the slices seed the planes that turn over. The rows of a song play the slices on + cheaply, so the slices seed the planes that turn over. Each reaches the dictionary as the + symbols its own plane is read in, so a seed and a stream name the same shape. The rows of a song play the slices on the channels it sounds, so those are the slices that seed it. Args: @@ -51,7 +56,11 @@ def phrases_from_project( channel_registers(channel, played, timer_table), pitches, ) - phrases.extend(Phrase(body=plane[:MAX_PHRASE_LENGTH]) for plane in planes.ordered if _turns_over(plane)) + phrases.extend( + Phrase(body=pack_plane(plane, PLANES[index].form, boundaries=NO_BOUNDARIES)[:MAX_PHRASE_LENGTH]) + for index, plane in zip(channel_indices(channel), planes, strict=True) + if _turns_over(plane) + ) return tuple(phrases) diff --git a/src/sampletones_player/compression/tokens/phrase.py b/src/sampletones_player/compression/tokens/phrase.py index 194e83e1e..237fb7604 100644 --- a/src/sampletones_player/compression/tokens/phrase.py +++ b/src/sampletones_player/compression/tokens/phrase.py @@ -8,14 +8,16 @@ class PhraseToken: """The plane plays a phrase from the table, shifted by ``transpose``, for ``ticks`` ticks. A count past the phrase's own length holds its final value onward, the way a note whose - envelope has finished keeps sounding, and a count short of it cuts the note off. + envelope has finished keeps sounding, and a count short of it cuts the note off. A token + playing the count the phrase itself carries states none, which is a byte it spares. """ phrase_id: int ticks: int transpose: int + default: bool @property def size(self) -> int: """The bytes the token takes.""" - return phrase_size(self.phrase_id, self.transpose) + return phrase_size(self.phrase_id, self.transpose, default=self.default) diff --git a/src/sampletones_player/compression/tokens/sizes.py b/src/sampletones_player/compression/tokens/sizes.py index 80d946333..a628efa23 100644 --- a/src/sampletones_player/compression/tokens/sizes.py +++ b/src/sampletones_player/compression/tokens/sizes.py @@ -17,19 +17,27 @@ def literal_size(length: int) -> int: return OPCODE_SIZE + length -def phrase_size(phrase_id: int, transpose: int) -> int: +def phrase_size( + phrase_id: int, + transpose: int, + *, + default: bool, +) -> int: """The bytes a phrase token takes. The opcode carries the phrase's id where the id is one of the cheap ones, and a further byte - names it beyond those. A count byte follows, and a shifted phrase carries the shift as well. + names it beyond those. A token stating a count of its own carries it in a byte, and one + playing the phrase's own count carries none. A shifted phrase carries the shift as well. Args: phrase_id: Position the phrase takes in the table. transpose: The shift every byte of the phrase is played at. + default: Whether the token plays the count the phrase carries. Returns: int: The bytes the token takes. """ escape = PHRASE_ESCAPE_SIZE if phrase_id >= PHRASE_ID_ESCAPE else 0 + count = 0 if default else PHRASE_COUNT_SIZE shift = TRANSPOSE_SIZE if transpose else 0 - return OPCODE_SIZE + PHRASE_COUNT_SIZE + escape + shift + return OPCODE_SIZE + count + escape + shift diff --git a/src/sampletones_player/compression/tokens/span.py b/src/sampletones_player/compression/tokens/span.py index bc67a7f5a..587b361bf 100644 --- a/src/sampletones_player/compression/tokens/span.py +++ b/src/sampletones_player/compression/tokens/span.py @@ -1,10 +1,13 @@ from typing import NamedTuple +from sampletones_player.compression.dictionary.table import PhraseTable from sampletones_player.specification.compression import ( + DEFAULT_COUNT_FLAG, OPCODE_SIZE, PHRASE_COUNT_SIZE, PHRASE_ESCAPE_SIZE, PHRASE_ID_ESCAPE, + PHRASE_ID_MASK, TOKEN_OPERAND_MASK, TOKEN_TAG_MASK, TRANSPOSE_SIZE, @@ -28,26 +31,39 @@ class TokenSpan(NamedTuple): ticks: int -def _phrase_span(data: bytes, position: int, *, transposed: bool) -> TokenSpan: - named = data[position] & TOKEN_OPERAND_MASK +def _phrase_span( + data: bytes, + position: int, + table: PhraseTable, + *, + transposed: bool, +) -> TokenSpan: + named = data[position] & PHRASE_ID_MASK + stated = not data[position] & DEFAULT_COUNT_FLAG escape = PHRASE_ESCAPE_SIZE if named == PHRASE_ID_ESCAPE else 0 - count = position + OPCODE_SIZE + escape + after = position + OPCODE_SIZE + escape + phrase_id = data[position + OPCODE_SIZE] if escape else named shift = TRANSPOSE_SIZE if transposed else 0 return TokenSpan( - size=OPCODE_SIZE + escape + PHRASE_COUNT_SIZE + shift, - ticks=data[count] + 1, + size=OPCODE_SIZE + escape + (PHRASE_COUNT_SIZE if stated else 0) + shift, + ticks=data[after] + 1 if stated else table[phrase_id].default, ) -def token_span(data: bytes, position: int) -> TokenSpan: +def token_span( + data: bytes, + position: int, + table: PhraseTable, +) -> TokenSpan: """Reads the token written at ``position``, answering what it takes and what it covers. Args: data: The plane's token stream. position: The byte the token's opcode lies at. + table: The dictionary the tokens name, for the count a phrase carries. Returns: - TokenSpan: The bytes the token takes and the ticks it covers. + TokenSpan: The bytes the token takes and the values it covers. """ operand = data[position] & TOKEN_OPERAND_MASK match TokenTag(data[position] & TOKEN_TAG_MASK): @@ -56,6 +72,6 @@ def token_span(data: bytes, position: int) -> TokenSpan: case TokenTag.LITERAL: return TokenSpan(size=OPCODE_SIZE + operand + 1, ticks=operand + 1) case TokenTag.PHRASE: - return _phrase_span(data, position, transposed=False) + return _phrase_span(data, position, table, transposed=False) case TokenTag.TRANSPOSED_PHRASE: - return _phrase_span(data, position, transposed=True) + return _phrase_span(data, position, table, transposed=True) diff --git a/src/sampletones_player/driver/binary/driver.bin b/src/sampletones_player/driver/binary/driver.bin index 697a28cda..b998206d4 100644 Binary files a/src/sampletones_player/driver/binary/driver.bin and b/src/sampletones_player/driver/binary/driver.bin differ diff --git a/src/sampletones_player/driver/image.py b/src/sampletones_player/driver/image.py index 79cbe1d22..6343006a1 100644 --- a/src/sampletones_player/driver/image.py +++ b/src/sampletones_player/driver/image.py @@ -45,7 +45,10 @@ def _validate_the_song_follows_the_code(self) -> DriverImage: @model_validator(mode="after") def _validate_the_routines_lie_in_the_code(self) -> DriverImage: - for name, address in (("init", self.addresses.init), ("play", self.addresses.play)): + for name, address in ( + ("init", self.addresses.init), + ("play", self.addresses.play), + ): if not self.addresses.load <= address < self.addresses.song: raise ValueError( f"the {name} routine lies at {address:#06x}, outside the code between " @@ -56,7 +59,10 @@ def _validate_the_routines_lie_in_the_code(self) -> DriverImage: @model_validator(mode="after") def _validate_the_image_leads_with_its_entry_points(self) -> DriverImage: - for name, address in (("init", self.addresses.init), ("play", self.addresses.play)): + for name, address in ( + ("init", self.addresses.init), + ("play", self.addresses.play), + ): opcode = self.code[address - self.addresses.load] if opcode != JUMP_ABSOLUTE_OPCODE: raise ValueError( diff --git a/src/sampletones_player/export/reports.py b/src/sampletones_player/export/reports.py index 0e3067a6c..1ef30f9b8 100644 --- a/src/sampletones_player/export/reports.py +++ b/src/sampletones_player/export/reports.py @@ -1,7 +1,10 @@ from sampletones_core.exports.progress import ExportProgress, ExportReporter from sampletones_core.exports.stage import ExportStage from sampletones_core.performance import WalkProgress, WalkReporter -from sampletones_player.compression.progress.report import CodecProgress, CodecReporter +from sampletones_player.compression.progress.report import ( + CodecProgress, + CodecReporter, +) UNMEASURED: None = None diff --git a/src/sampletones_player/export/writer.py b/src/sampletones_player/export/writer.py index c66c7669d..a9987ba90 100644 --- a/src/sampletones_player/export/writer.py +++ b/src/sampletones_player/export/writer.py @@ -9,7 +9,11 @@ from sampletones_player.builder import song_from_project, song_from_sample from sampletones_player.driver.image import DriverImage from sampletones_player.export.program import NSFProgram -from sampletones_player.export.reports import UNMEASURED, codec_reporter, walk_reporter +from sampletones_player.export.reports import ( + UNMEASURED, + codec_reporter, + walk_reporter, +) from sampletones_player.nsf.file import write_nsf from sampletones_player.song import Song diff --git a/src/sampletones_player/nsf/layout.py b/src/sampletones_player/nsf/layout.py index 90f212512..39d6f5c5c 100644 --- a/src/sampletones_player/nsf/layout.py +++ b/src/sampletones_player/nsf/layout.py @@ -3,18 +3,20 @@ from dataclasses import dataclass from typing import Final, Optional, Sequence, Tuple -from sampletones_player.compression.decode import decode_planes from sampletones_player.compression.dictionary.table import PhraseTable from sampletones_player.compression.planes.order import PlaneOrder from sampletones_player.song import Song from sampletones_player.specification.compression import ( + PHRASE_DEFAULT_SIZE, PHRASE_LENGTH_SIZE, PHRASE_TABLE_COUNT_SIZE, PHRASE_TABLE_ENTRY_SIZE, ) -from sampletones_player.specification.song import ABSENT_STREAM, SONG_HEADER_SIZE +from sampletones_player.specification.song import ( + ABSENT_STREAM, + SONG_HEADER_SIZE, +) -FIRST_TICK: Final[int] = 0 NAME_SEPARATOR: Final[str] = "_" @@ -80,7 +82,7 @@ def of(cls, song: Song) -> SongLayout: """ phrases = song.planes.phrases streams = song.planes.streams - body_sizes = [PHRASE_LENGTH_SIZE + phrase.length for phrase in phrases.phrases] + body_sizes = [PHRASE_LENGTH_SIZE + PHRASE_DEFAULT_SIZE + phrase.length for phrase in phrases.phrases] stream_sizes = [len(stream) for stream in streams] timer_table = SONG_HEADER_SIZE @@ -89,8 +91,7 @@ def of(cls, song: Song) -> SongLayout: stream_start = bodies + sum(body_sizes) stream_offsets = _running(stream_start, stream_sizes) - returned = FIRST_TICK if song.loop_tick is None else song.loop_tick - entered = song.planes.entries(decode_planes(song.planes).positions(returned)) + entered = song.planes.loop_entries return cls( timer_table=timer_table, phrase_table=phrase_table, diff --git a/src/sampletones_player/nsf/song.py b/src/sampletones_player/nsf/song.py index ba76470fc..84826f084 100644 --- a/src/sampletones_player/nsf/song.py +++ b/src/sampletones_player/nsf/song.py @@ -1,12 +1,17 @@ +from typing import Final + from sampletones_core.formats.binary import BinaryWriter from sampletones_player.compression.dictionary.table import PhraseTable from sampletones_player.compression.pitch import PitchTable from sampletones_player.compression.planes.order import PlaneOrder from sampletones_player.nsf.layout import SongLayout from sampletones_player.song import Song +from sampletones_player.specification.compression import NO_DEFAULT_COUNT from sampletones_player.specification.song import MAX_BLOCK_OFFSET, NO_LOOP from sampletones_shared.exceptions import SongTooLargeError +STATED_BEYOND_THE_FIRST: Final[int] = 1 + def _write_header( writer: BinaryWriter, @@ -42,6 +47,7 @@ def _write_phrase_table( for phrase in phrases.phrases: writer.write_uint8(phrase.length) + writer.write_uint8(max(phrase.default - STATED_BEYOND_THE_FIRST, NO_DEFAULT_COUNT)) writer.write_bytes(phrase.body) diff --git a/src/sampletones_player/registers/pulse.py b/src/sampletones_player/registers/pulse.py index ddf9289cb..99eeb5c97 100644 --- a/src/sampletones_player/registers/pulse.py +++ b/src/sampletones_player/registers/pulse.py @@ -6,7 +6,10 @@ from sampletones_core.exporters.implementation.pulse import PulseExporter from sampletones_core.instructions import PulseInstruction -from sampletones_player.registers.dividers import anchored_pitches, bent_dividers +from sampletones_player.registers.dividers import ( + anchored_pitches, + bent_dividers, +) from sampletones_player.registers.hold import hold from sampletones_player.registers.tone import ToneRegisters from sampletones_player.specification.registers import ( diff --git a/src/sampletones_player/registers/triangle.py b/src/sampletones_player/registers/triangle.py index f7c997775..ed1affd45 100644 --- a/src/sampletones_player/registers/triangle.py +++ b/src/sampletones_player/registers/triangle.py @@ -6,7 +6,10 @@ from sampletones_core.exporters.implementation.triangle import TriangleExporter from sampletones_core.instructions import TriangleInstruction -from sampletones_player.registers.dividers import anchored_pitches, bent_dividers +from sampletones_player.registers.dividers import ( + anchored_pitches, + bent_dividers, +) from sampletones_player.registers.hold import hold from sampletones_player.registers.tone import ToneRegisters from sampletones_player.specification.registers import ( @@ -16,6 +19,7 @@ TRIANGLE_SILENT_RELOAD, TRIANGLE_SOUNDING_RELOAD, ) +from sampletones_shared.constants.music import LIMIT_MIN_PITCH class TriangleRegisters(ToneRegisters): @@ -39,7 +43,9 @@ def from_instructions( reload every frame and the note last as long as the ticks do. The timer carries the note's divider moved by the frame's bend, and the channel sounds an - octave below the pitch it names — the same octave a rendered triangle sounds. + octave below the pitch it names — the same octave a rendered triangle sounds. A resting + tick holds the divider the channel last sounded, and the lowest pitch's before it sounds + at all, so the pitch a rest leaves unstated is one that can be reached again. Args: instructions: The channel's per-tick instructions. @@ -49,19 +55,29 @@ def from_instructions( List[TriangleRegisters]: One register set per tick, including the closing release tick. """ _, pitches, volumes = TriangleExporter.extract_data(instructions) - dividers = bent_dividers(pitches, TriangleExporter.read_timer_offsets(instructions), timer_table) + dividers = bent_dividers( + pitches, + TriangleExporter.read_timer_offsets(instructions), + timer_table, + ) anchors = anchored_pitches(pitches, dividers, timer_table) registers: List[TriangleRegisters] = [] + timer = timer_table[LIMIT_MIN_PITCH] + anchor = LIMIT_MIN_PITCH for index, volume in enumerate(volumes): - timer = hold(dividers, index) - reload_value = TRIANGLE_SOUNDING_RELOAD if volume > 0 else TRIANGLE_SILENT_RELOAD + sounding = volume > 0 + if sounding: + timer = hold(dividers, index) + anchor = hold(anchors, index) + + reload_value = TRIANGLE_SOUNDING_RELOAD if sounding else TRIANGLE_SILENT_RELOAD registers.append( cls( linear_counter=TRIANGLE_COUNTER_CONTROL | reload_value, timer_low=timer & MAX_REGISTER_VALUE, timer_high=timer >> TIMER_HIGH_SHIFT, - anchor=hold(anchors, index), + anchor=anchor, ) ) diff --git a/src/sampletones_player/specification/channels.py b/src/sampletones_player/specification/channels.py index 8c7c00cbd..9d86de7da 100644 --- a/src/sampletones_player/specification/channels.py +++ b/src/sampletones_player/specification/channels.py @@ -18,6 +18,10 @@ CHANNEL_REGISTER_ADDRESSES: Final[Dict[ChannelName, Tuple[int, ...]]] = { ChannelName.PULSE1: (PULSE1_CONTROL, PULSE1_TIMER_LOW, PULSE1_TIMER_HIGH), ChannelName.PULSE2: (PULSE2_CONTROL, PULSE2_TIMER_LOW, PULSE2_TIMER_HIGH), - ChannelName.TRIANGLE: (TRIANGLE_LINEAR_COUNTER, TRIANGLE_TIMER_LOW, TRIANGLE_TIMER_HIGH), + ChannelName.TRIANGLE: ( + TRIANGLE_LINEAR_COUNTER, + TRIANGLE_TIMER_LOW, + TRIANGLE_TIMER_HIGH, + ), ChannelName.NOISE: (NOISE_CONTROL, NOISE_PERIOD), } diff --git a/src/sampletones_player/specification/compression.py b/src/sampletones_player/specification/compression.py index a8e301da5..c2a96c2df 100644 --- a/src/sampletones_player/specification/compression.py +++ b/src/sampletones_player/specification/compression.py @@ -2,7 +2,6 @@ from math import ceil from typing import Final -from sampletones_core.constants.enums import TONE_CHANNELS, ChannelName from sampletones_player.specification.binary import ( BYTE_VALUES, MAX_BYTE_VALUE, @@ -19,6 +18,10 @@ class TokenTag(IntEnum): LITERAL: The plane takes the bytes that follow, one per tick. PHRASE: The plane plays a phrase from the table at the pitch it was stored at. TRANSPOSED_PHRASE: The plane plays a phrase shifted by the signed byte that follows. + + A phrase opcode spends the top bit of its operand on whether the token states a count of its + own or plays the count the phrase itself carries, so the ids it names outright are the lower + half of what a hold or a literal counts. """ HOLD = 0x00 @@ -39,7 +42,9 @@ class TokenTag(IntEnum): MAX_LITERAL_BYTES: Final[int] = TOKEN_OPERAND_MASK + 1 MAX_PHRASE_TICKS: Final[int] = BYTE_VALUES -PHRASE_ID_ESCAPE: Final[int] = TOKEN_OPERAND_MASK +DEFAULT_COUNT_FLAG: Final[int] = (TOKEN_OPERAND_MASK + 1) >> 1 +PHRASE_ID_MASK: Final[int] = DEFAULT_COUNT_FLAG - 1 +PHRASE_ID_ESCAPE: Final[int] = PHRASE_ID_MASK CHEAP_PHRASE_IDS: Final[int] = PHRASE_ID_ESCAPE MAX_PHRASE_IDS: Final[int] = MAX_BYTE_VALUE MAX_PHRASE_LENGTH: Final[int] = MAX_BYTE_VALUE @@ -47,14 +52,11 @@ class TokenTag(IntEnum): PHRASE_TABLE_COUNT_SIZE: Final[int] = ceil(MAX_PHRASE_IDS.bit_length() / BITS_PER_BYTE) PHRASE_TABLE_ENTRY_SIZE: Final[int] = WORD_SIZE PHRASE_LENGTH_SIZE: Final[int] = 1 +PHRASE_DEFAULT_SIZE: Final[int] = 1 +NO_DEFAULT_COUNT: Final[int] = 0 INITIAL_PLANE_VALUE: Final[int] = 0 BEND_FLAG: Final[int] = 0x80 PITCH_INDEX_MASK: Final[int] = BEND_FLAG - 1 -PLANES_PER_CHANNEL: Final[int] = 2 -TONE_PLANES_PER_CHANNEL: Final[int] = PLANES_PER_CHANNEL + 1 -PLANE_COUNT: Final[int] = sum( - TONE_PLANES_PER_CHANNEL if channel in TONE_CHANNELS else PLANES_PER_CHANNEL for channel in ChannelName.items() -) -PLANE_STATE_SIZE: Final[int] = 8 +PLANE_STATE_SIZE: Final[int] = 10 diff --git a/src/sampletones_player/specification/planes.py b/src/sampletones_player/specification/planes.py new file mode 100644 index 000000000..e7e15087e --- /dev/null +++ b/src/sampletones_player/specification/planes.py @@ -0,0 +1,314 @@ +from __future__ import annotations + +from enum import StrEnum +from typing import Final, Tuple + +from pydantic import BaseModel, ConfigDict, Field, model_validator + +from sampletones_core.constants.enums import ChannelName +from sampletones_core.constants.general import ( + MAX_DUTY_CYCLE, + MAX_PERIOD, + MAX_VOLUME, +) +from sampletones_player.specification.binary import MAX_BYTE_VALUE +from sampletones_player.specification.compression import PITCH_INDEX_MASK +from sampletones_player.specification.registers import ( + DUTY_CYCLE_SHIFT, + NOISE_MODE_SHIFT, + SUSTAINED_LEVEL, +) + +SINGLE_TICK: Final[int] = 1 +NO_BITS: Final[int] = 0 +COUNT_STEP: Final[int] = 0x10 +SILENT_PITCH_INDEX: Final[int] = PITCH_INDEX_MASK + + +class PlaneForm(BaseModel): + """How a plane's byte divides between the value its register reads and a repeat count. + + A channel's register reads some of a byte's bits and ignores the rest: a pulse channel's + volume is four bits under two the hardware wants set, and a noise period is four bits with + three above it that say nothing. A plane writes its value in the bits that reach the register + and counts the ticks that value repeats for in the ones left over, so a run costs one byte + however long it rests. + + The division is a mask, which is what keeps the reading cheap: the register byte is the + symbol masked and ored with the bits the hardware fixes, and one repeat is a subtraction of + ``COUNT_STEP``. A plane whose values need every bit states a full ``value_mask``, carries no + count, and reads byte for byte. + + Attributes: + value_mask: The bits the value occupies. + value_or: The bits the register fixes, which every value of the plane carries. + """ + + model_config = ConfigDict(extra="forbid", frozen=True) + + value_mask: int = Field(..., ge=0, le=MAX_BYTE_VALUE) + value_or: int = Field(..., ge=0, le=MAX_BYTE_VALUE) + + @model_validator(mode="after") + def _validate_the_fixed_bits_lie_outside_the_value(self) -> PlaneForm: + if self.value_or & self.value_mask: + raise ValueError( + f"the bits a register fixes lie outside the value's own, and {self.value_or:#04x} " + f"meets {self.value_mask:#04x}" + ) + + return self + + @model_validator(mode="after") + def _validate_a_count_steps_from_one_place(self) -> PlaneForm: + if self.counts and self.count_mask & -self.count_mask != COUNT_STEP: + raise ValueError( + f"a repeat count steps from {COUNT_STEP:#04x}, and {self.count_mask:#04x} steps " + f"from {self.count_mask & -self.count_mask:#04x}" + ) + + return self + + @model_validator(mode="after") + def _validate_the_count_occupies_one_run_of_bits(self) -> PlaneForm: + counted = self.count_mask // COUNT_STEP if self.counts else NO_BITS + if counted & (counted + 1): + raise ValueError(f"a repeat count occupies one run of bits, and {self.count_mask:#04x} breaks apart") + + return self + + @property + def count_mask(self) -> int: + """The bits a repeat count occupies.""" + return MAX_BYTE_VALUE ^ self.value_mask + + @property + def counts(self) -> bool: + """Whether the plane's byte has room to count a repeat.""" + return bool(self.count_mask) + + @property + def repeats(self) -> int: + """The ticks one symbol covers at most.""" + if not self.counts: + return SINGLE_TICK + + return self.count_mask // COUNT_STEP + SINGLE_TICK + + def symbol( + self, + value: int, + repeats: int, + ) -> int: + """The byte a plane writes for ``value`` sounding ``repeats`` ticks running. + + Args: + value: The register byte the ticks play. + repeats: The ticks the value sounds for. + + Returns: + int: The symbol. + + Raises: + ValueError: If the value carries fixed bits other than the form's, or the repeats + reach past what one symbol counts. + """ + if value & self.count_mask != self.value_or: + raise ValueError( + f"a value of this plane carries {self.value_or:#04x} in the bits the count takes, " + f"and {value:#04x} carries {value & self.count_mask:#04x}" + ) + + if not SINGLE_TICK <= repeats <= self.repeats: + raise ValueError(f"one symbol counts {SINGLE_TICK} through {self.repeats} ticks, and this counts {repeats}") + + return (value & self.value_mask) | (repeats - SINGLE_TICK) * COUNT_STEP + + def value(self, symbol: int) -> int: + """The register byte ``symbol`` plays.""" + return (symbol & self.value_mask) | self.value_or + + def repeated(self, symbol: int) -> int: + """The ticks ``symbol`` covers.""" + if not self.counts: + return SINGLE_TICK + + return (symbol & self.count_mask) // COUNT_STEP + SINGLE_TICK + + def fits(self, plane: bytes) -> bool: + """Whether every value the plane plays carries the bits this form fixes. + + Args: + plane: The values the plane plays. + + Returns: + bool: Whether the form reads the plane. + """ + return all(value & self.count_mask == self.value_or for value in plane) + + +WHOLE_BYTE: Final[PlaneForm] = PlaneForm(value_mask=MAX_BYTE_VALUE, value_or=NO_BITS) + +DUTY_CYCLE_FIELD: Final[int] = MAX_DUTY_CYCLE << DUTY_CYCLE_SHIFT +VOLUME_FIELD: Final[int] = MAX_VOLUME +PERIOD_FIELD: Final[int] = MAX_PERIOD +NOISE_MODE_FIELD: Final[int] = 1 << NOISE_MODE_SHIFT + +PULSE_CONTROL_FORM: Final[PlaneForm] = PlaneForm( + value_mask=DUTY_CYCLE_FIELD | VOLUME_FIELD, + value_or=SUSTAINED_LEVEL, +) +NOISE_CONTROL_FORM: Final[PlaneForm] = PlaneForm( + value_mask=VOLUME_FIELD, + value_or=SUSTAINED_LEVEL, +) +NOISE_VALUE_FORM: Final[PlaneForm] = PlaneForm( + value_mask=NOISE_MODE_FIELD | PERIOD_FIELD, + value_or=NO_BITS, +) + + +class PlaneRole(StrEnum): + """The part of a channel's tick one plane carries. + + Attributes: + CONTROL: How the channel sounds — its timbre, and its volume where it has one. + VALUE: What the channel sounds — a pitch index, or the noise channel's period byte. + BEND: How far a tick stands from the divider the pitch it names sounds at. + """ + + CONTROL = "control" + VALUE = "value" + BEND = "bend" + + +class Plane(BaseModel): + """One byte series a song block writes, named by the channel and the part it carries. + + Attributes: + channel: The channel the plane belongs to. + role: The part of the channel's tick the plane carries. + form: How the plane's byte divides between its value and a repeat count. + seeded: The register byte the plane stands at before its first token, which the driver + writes when it starts a song and a plane holding it throughout takes no stream. + """ + + model_config = ConfigDict(extra="forbid", frozen=True) + + channel: ChannelName + role: PlaneRole + form: PlaneForm + seeded: int = Field(..., ge=0, le=MAX_BYTE_VALUE) + + @model_validator(mode="after") + def _validate_the_seeded_value_reads_as_the_plane_does(self) -> Plane: + if not self.form.fits(bytes((self.seeded,))): + raise ValueError( + f"a plane stands at a value its own form reads, and {self.name} stands at {self.seeded:#04x}" + ) + + return self + + @property + def name(self) -> str: + """The name the song block writes the plane under.""" + return f"{self.channel.value}_{self.role.value}" + + @property + def spans_flagged_ticks(self) -> bool: + """Whether the plane holds a value per flagged tick rather than one per tick.""" + return self.role is PlaneRole.BEND + + def idles(self, played: bytes) -> bool: + """Whether the plane holds the value it is seeded to throughout, so it takes no stream. + + Args: + played: The values the plane plays. + + Returns: + bool: Whether the plane stands where the driver seeds it. + """ + return set(played) <= {self.seeded} + + +def _tone( + channel: ChannelName, + role: PlaneRole, +) -> Plane: + return Plane(channel=channel, role=role, form=WHOLE_BYTE, seeded=NO_BITS) + + +PLANES: Final[Tuple[Plane, ...]] = ( + Plane( + channel=ChannelName.PULSE1, + role=PlaneRole.CONTROL, + form=PULSE_CONTROL_FORM, + seeded=SUSTAINED_LEVEL, + ), + _tone(ChannelName.PULSE1, PlaneRole.VALUE), + _tone(ChannelName.PULSE1, PlaneRole.BEND), + Plane( + channel=ChannelName.PULSE2, + role=PlaneRole.CONTROL, + form=PULSE_CONTROL_FORM, + seeded=SUSTAINED_LEVEL, + ), + _tone(ChannelName.PULSE2, PlaneRole.VALUE), + _tone(ChannelName.PULSE2, PlaneRole.BEND), + Plane( + channel=ChannelName.TRIANGLE, + role=PlaneRole.VALUE, + form=WHOLE_BYTE, + seeded=SILENT_PITCH_INDEX, + ), + _tone(ChannelName.TRIANGLE, PlaneRole.BEND), + Plane( + channel=ChannelName.NOISE, + role=PlaneRole.CONTROL, + form=NOISE_CONTROL_FORM, + seeded=SUSTAINED_LEVEL, + ), + Plane( + channel=ChannelName.NOISE, + role=PlaneRole.VALUE, + form=NOISE_VALUE_FORM, + seeded=NO_BITS, + ), +) +PLANE_COUNT: Final[int] = len(PLANES) +PLANE_NAMES: Final[Tuple[str, ...]] = tuple(plane.name for plane in PLANES) + + +def channel_indices(channel: ChannelName) -> Tuple[int, ...]: + """Where ``channel``'s own planes stand in the song block, in the order it writes them. + + Args: + channel: The channel to read. + + Returns: + Tuple[int, ...]: The positions that channel's planes take. + """ + return tuple(index for index, plane in enumerate(PLANES) if plane.channel is channel) + + +def plane_index( + channel: ChannelName, + role: PlaneRole, +) -> int: + """Where the plane ``channel`` writes for ``role`` stands in the song block. + + Args: + channel: The channel the plane belongs to. + role: The part of the channel's tick the plane carries. + + Returns: + int: The position the plane takes. + + Raises: + ValueError: If the channel writes no plane for that role. + """ + for index, plane in enumerate(PLANES): + if plane.channel is channel and plane.role is role: + return index + + raise ValueError(f"the {channel.value} channel writes no {role.value} plane") diff --git a/src/sampletones_player/specification/song.py b/src/sampletones_player/specification/song.py index 5a81ecbc5..f6e03ad09 100644 --- a/src/sampletones_player/specification/song.py +++ b/src/sampletones_player/specification/song.py @@ -1,7 +1,7 @@ from typing import Final from sampletones_player.specification.binary import WORD_SIZE -from sampletones_player.specification.compression import PLANE_COUNT +from sampletones_player.specification.planes import PLANE_COUNT STEP_WHOLE_OFFSET: Final[int] = 0 STEP_FRACTION_OFFSET: Final[int] = STEP_WHOLE_OFFSET + 1 diff --git a/src/sampletones_tools/codec/report/encoding.py b/src/sampletones_tools/codec/report/encoding.py index 073ea8c7f..268f23394 100644 --- a/src/sampletones_tools/codec/report/encoding.py +++ b/src/sampletones_tools/codec/report/encoding.py @@ -2,7 +2,7 @@ from time import process_time from typing import Final, List, Sequence, Tuple -from sampletones_player.compression.absent import is_absent +from sampletones_core.constants.enums import PULSE_CHANNELS from sampletones_player.compression.compressed import CompressedPlanes from sampletones_player.compression.dictionary.table import phrase_table from sampletones_player.compression.encode import STREAM_START, encode_planes @@ -11,10 +11,10 @@ from sampletones_player.compression.matches.matcher import PhraseMatcher from sampletones_player.compression.options import CodecOptions from sampletones_player.compression.parse.plane import parse_plane -from sampletones_player.compression.planes.channel import TonePlanes from sampletones_player.compression.planes.song import SongPlanes from sampletones_player.compression.scheme import CompressionScheme from sampletones_player.registers.streams import ChannelStreams +from sampletones_player.specification.planes import NO_BITS, PLANES, PlaneRole from sampletones_player.specification.registers import DUTY_CYCLE_SHIFT from sampletones_tools.codec.report.corpus import CorpusEntry from sampletones_tools.codec.report.rows import ReportRow @@ -203,35 +203,46 @@ def _measured_row( def _register_planes(streams: ChannelStreams) -> Tuple[bytes, ...]: - return tuple( - bytes(tick.values[register] for tick in stream) - for stream in streams.padded - for register in range(len(stream[0].values)) - ) + """Every register as a plane, the ones a channel leaves at the value it resets to left out.""" + written: List[bytes] = [] + for stream in streams.padded: + for register in range(len(stream[0].values)): + plane = bytes(tick.values[register] for tick in stream) + if set(plane) != {NO_BITS}: + written.append(plane) + + return tuple(written) def _split_control_planes(planes: SongPlanes) -> Tuple[bytes, ...]: - """Every plane with each pulse channel's control split into duty and volume.""" + """Every plane the block writes, with each pulse channel's control split into duty and volume. + + A plane standing at the value it is seeded to throughout takes no stream, so it takes no + bytes here either, and a control plane splits into the two series a split layout would store. + + Args: + planes: The song's planes. + + Returns: + Tuple[bytes, ...]: The series the layout stores, in the order the block writes them. + """ split: List[bytes] = [] - for channels in planes.ordered: - if channels in (planes.pulse1, planes.pulse2): - split.append(bytes(control >> DUTY_CYCLE_SHIFT for control in channels.control)) - split.append(bytes(control & CONTROL_LEVEL_MASK for control in channels.control)) - else: - split.append(channels.control) + for plane, played in zip(PLANES, planes.planes, strict=True): + if plane.idles(played): + continue - split.append(channels.value) - match channels: - case TonePlanes(): - split.append(channels.bend) + if plane.role is PlaneRole.CONTROL and plane.channel in PULSE_CHANNELS: + split.append(bytes(control >> DUTY_CYCLE_SHIFT for control in played)) + split.append(bytes(control & CONTROL_LEVEL_MASK for control in played)) + else: + split.append(played) return tuple(split) def _coded_size(planes: Sequence[bytes], options: CodecOptions) -> int: - """The bytes the planes take under holds and literals, an absent plane taking none.""" - present = [plane for plane in planes if not is_absent(plane)] - cache = MatchCache(PlaneIndex.from_plane(plane) for plane in present) + """The bytes the planes take under holds and literals.""" + cache = MatchCache(PlaneIndex.from_plane(plane) for plane in planes) table = phrase_table(()) entries = frozenset({STREAM_START}) return sum( @@ -240,5 +251,5 @@ def _coded_size(planes: Sequence[bytes], options: CodecOptions) -> int: options, entries, ).size - for plane in range(len(present)) + for plane in range(len(planes)) ) diff --git a/src/sampletones_tools/codec/report/session.py b/src/sampletones_tools/codec/report/session.py index 5a8d66019..c846e428c 100644 --- a/src/sampletones_tools/codec/report/session.py +++ b/src/sampletones_tools/codec/report/session.py @@ -3,7 +3,8 @@ from typing import Final, Sequence, Tuple from sampletones_player.driver.image import DriverImage -from sampletones_player.specification.compression import PLANE_COUNT, PLANE_STATE_SIZE +from sampletones_player.specification.compression import PLANE_STATE_SIZE +from sampletones_player.specification.planes import PLANE_COUNT from sampletones_tools.codec.report.corpus import CorpusEntry, corpus_entries from sampletones_tools.codec.report.encoding import Encoding, encode_corpus, report_rows from sampletones_tools.codec.report.rows import report_table, write_markdown diff --git a/src/sampletones_tools/codec/study/accounting/dictionary.py b/src/sampletones_tools/codec/study/accounting/dictionary.py index 1307bd4b2..03d83e377 100644 --- a/src/sampletones_tools/codec/study/accounting/dictionary.py +++ b/src/sampletones_tools/codec/study/accounting/dictionary.py @@ -1,16 +1,12 @@ -from collections import Counter -from typing import Dict, Final, Iterable +from typing import Final from sampletones_player.compression.dictionary.table import PhraseTable -from sampletones_player.specification.compression import PHRASE_COUNT_SIZE from sampletones_tools.codec.study.accounting.finding import NOTHING, Finding from sampletones_tools.codec.study.accounting.runs import runs -from sampletones_tools.codec.study.accounting.tokens import ReadToken REPEATED: Final[int] = 2 MIN_RLE_RUN: Final[int] = 3 RLE_RUN_SIZE: Final[int] = 2 -DEFAULT_COUNT_SIZE: Final[int] = 1 def plateaus(table: PhraseTable) -> Finding: @@ -32,35 +28,3 @@ def plateaus(table: PhraseTable) -> Finding: finding += Finding(run - 1, max(0, run - RLE_RUN_SIZE) if run >= MIN_RLE_RUN else 0) return finding - - -def default_counts( - table: PhraseTable, - tokens: Iterable[ReadToken], -) -> Finding: - """What a default count stored with each phrase would spare on the count bytes (H4). - - Every phrase token carries the ticks it plays for. A phrase played mostly for one length - could state that length once in its body, and a token playing it that long could leave the - count byte out. - - Args: - table: The dictionary. - tokens: Every token of every plane. - - Returns: - Finding: The count bytes phrase tokens carry, and the bytes a default would spare. - """ - counts: Dict[int, Counter[int]] = {phrase_id: Counter() for phrase_id in range(len(table))} - for token in tokens: - if token.phrase_id is not None: - counts[token.phrase_id][token.ticks] += 1 - - targeted = PHRASE_COUNT_SIZE * sum(sum(counter.values()) for counter in counts.values()) - saving = 0 - for counter in counts.values(): - if counter: - (_, modal), *_ = counter.most_common(1) - saving += max(0, PHRASE_COUNT_SIZE * modal - DEFAULT_COUNT_SIZE) - - return Finding(targeted, saving) diff --git a/src/sampletones_tools/codec/study/accounting/fixed.py b/src/sampletones_tools/codec/study/accounting/fixed.py index c168e59b5..3a2a8f2a6 100644 --- a/src/sampletones_tools/codec/study/accounting/fixed.py +++ b/src/sampletones_tools/codec/study/accounting/fixed.py @@ -1,13 +1,14 @@ from dataclasses import dataclass from typing import Final +from sampletones_core.constants.enums import TONE_CHANNELS from sampletones_player.compression.compressed import CompressedPlanes from sampletones_player.compression.planes.flags import pitch_index from sampletones_player.specification.binary import WORD_SIZE from sampletones_player.specification.compression import ( PHRASE_TABLE_ENTRY_SIZE, - PLANE_COUNT, ) +from sampletones_player.specification.planes import PLANE_COUNT, PLANES, PlaneRole from sampletones_player.specification.song import SONG_HEADER_SIZE from sampletones_tools.codec.study.accounting.finding import Finding from sampletones_tools.codec.study.corpus.song import StudySong @@ -54,8 +55,12 @@ def fixed_overheads( Returns: FixedOverheads: The fixed bytes, part by part. """ - tones = (song.planes.pulse1, song.planes.pulse2, song.planes.triangle) - highest = max(pitch_index(value) for channel in tones for value in channel.value) + tones = ( + song.planes.planes[index] + for index, plane in enumerate(PLANES) + if plane.role is PlaneRole.VALUE and plane.channel in TONE_CHANNELS + ) + highest = max(pitch_index(value) for plane in tones for value in plane) return FixedOverheads( header=SONG_HEADER_SIZE, pitch_table=len(song.pitches.data), diff --git a/src/sampletones_tools/codec/study/accounting/rows.py b/src/sampletones_tools/codec/study/accounting/rows.py index ff0e9c832..b7c31b957 100644 --- a/src/sampletones_tools/codec/study/accounting/rows.py +++ b/src/sampletones_tools/codec/study/accounting/rows.py @@ -4,7 +4,7 @@ from sampletones_player.compression.compressed import CompressedPlanes from sampletones_player.compression.planes.order import PlaneOrder from sampletones_tools.codec.study.accounting.coincident import coincident_starts -from sampletones_tools.codec.study.accounting.dictionary import default_counts, plateaus +from sampletones_tools.codec.study.accounting.dictionary import plateaus from sampletones_tools.codec.study.accounting.finding import NOTHING, Finding from sampletones_tools.codec.study.accounting.fixed import fixed_overheads from sampletones_tools.codec.study.accounting.pairs import ramps_in_literals, set_holds @@ -16,7 +16,6 @@ ("H1", "hold chains"), ("H2", "plateaus in bodies"), ("H3", "set-then-hold"), - ("H4", "default counts"), ("H5", "coincident starts"), ("H6", "ramps in literals"), ("H7", "fixed tables"), @@ -114,18 +113,16 @@ def account( AccountingRow: The shares and the findings. """ tokens: Dict[str, Tuple[ReadToken, ...]] = { - name: read_tokens(stream) for name, stream in zip(PlaneOrder.names(), compressed.streams) + name: read_tokens(stream, compressed.phrases) for name, stream in zip(PlaneOrder.names(), compressed.streams) } planes = tuple( plane_shares(name, plane, tokens[name]) for name, plane in zip(PlaneOrder.names(), measurement.song.planes.planes) ) - every_token = [token for name in PlaneOrder.names() for token in tokens[name]] findings = ( sum((plane.hold_chains for plane in planes), NOTHING), plateaus(compressed.phrases), sum((set_holds(tokens[name]) for name in PlaneOrder.names()), NOTHING), - default_counts(compressed.phrases, every_token), coincident_starts(tokens), sum((ramps_in_literals(tokens[name]) for name in PlaneOrder.names()), NOTHING), fixed_overheads(measurement.song, compressed).finding, diff --git a/src/sampletones_tools/codec/study/accounting/tokens.py b/src/sampletones_tools/codec/study/accounting/tokens.py index 73deed04f..90734a276 100644 --- a/src/sampletones_tools/codec/study/accounting/tokens.py +++ b/src/sampletones_tools/codec/study/accounting/tokens.py @@ -1,11 +1,14 @@ from typing import List, NamedTuple, Optional, Tuple +from sampletones_player.compression.dictionary.table import PhraseTable from sampletones_player.compression.tokens.span import token_span from sampletones_player.specification.compression import ( + DEFAULT_COUNT_FLAG, OPCODE_SIZE, PHRASE_COUNT_SIZE, PHRASE_ESCAPE_SIZE, PHRASE_ID_ESCAPE, + PHRASE_ID_MASK, TOKEN_OPERAND_MASK, TOKEN_TAG_MASK, TokenTag, @@ -47,20 +50,27 @@ def _phrase_operands( transposed: bool, ) -> Tuple[int, int]: after = position + OPCODE_SIZE - phrase_id = operand - if operand == PHRASE_ID_ESCAPE: + phrase_id = operand & PHRASE_ID_MASK + if phrase_id == PHRASE_ID_ESCAPE: phrase_id = stream[after] after += PHRASE_ESCAPE_SIZE - transpose = stream[after + PHRASE_COUNT_SIZE] if transposed else 0 + if not operand & DEFAULT_COUNT_FLAG: + after += PHRASE_COUNT_SIZE + + transpose = stream[after] if transposed else 0 return phrase_id, transpose -def read_tokens(stream: bytes) -> Tuple[ReadToken, ...]: +def read_tokens( + stream: bytes, + table: PhraseTable, +) -> Tuple[ReadToken, ...]: """Reads a plane's stream back as the tokens it was written from. Args: stream: The plane's token stream. + table: The dictionary the tokens name. Returns: Tuple[ReadToken, ...]: The tokens in the order they are read. @@ -69,7 +79,7 @@ def read_tokens(stream: bytes) -> Tuple[ReadToken, ...]: position = 0 tick = 0 while position < len(stream): - span = token_span(stream, position) + span = token_span(stream, position, table) tag = TokenTag(stream[position] & TOKEN_TAG_MASK) operand = stream[position] & TOKEN_OPERAND_MASK payload = b"" diff --git a/src/sampletones_tools/codec/study/corpus/song.py b/src/sampletones_tools/codec/study/corpus/song.py index ad8d5c6a0..3f12b9c31 100644 --- a/src/sampletones_tools/codec/study/corpus/song.py +++ b/src/sampletones_tools/codec/study/corpus/song.py @@ -6,7 +6,6 @@ from sampletones_core.constants.enums import ChannelName from sampletones_player.compression.dictionary.phrase import Phrase from sampletones_player.compression.pitch import PitchTable -from sampletones_player.compression.planes.channel import ChannelPlanes from sampletones_player.compression.planes.song import SongPlanes @@ -31,12 +30,12 @@ class StudySlice: Attributes: channel: The channel the slice plays on. - planes: The slice's planes as the production codec separates them. + planes: The slice's planes as the production codec separates them, in block order. notes: The note each of its ticks names, empty on the noise channel. """ channel: ChannelName - planes: ChannelPlanes + planes: Tuple[bytes, ...] notes: bytes diff --git a/src/sampletones_tools/codec/study/layouts/encode.py b/src/sampletones_tools/codec/study/layouts/encode.py index e300b664b..816a13283 100644 --- a/src/sampletones_tools/codec/study/layouts/encode.py +++ b/src/sampletones_tools/codec/study/layouts/encode.py @@ -3,6 +3,9 @@ from sampletones_player.compression.decode import decode_plane from sampletones_player.compression.encode import encode_streams from sampletones_player.compression.options import EVERY_LAYER +from sampletones_player.compression.planes.symbols import pack_plane +from sampletones_player.specification.planes import PLANES +from sampletones_player.specification.song import SONG_HEADER_SIZE from sampletones_tools.codec.study.corpus.song import StudySong from sampletones_tools.codec.study.layouts.layout import PlaneLayout from sampletones_tools.codec.study.layouts.planes import layout_planes, layout_seeds @@ -25,23 +28,29 @@ def encode_layout( """ planes = layout_planes(song, layout) seeds = layout_seeds(song.slices, song.pitches, layout) + packed = tuple( + b"" if plane.idles(played) else pack_plane(played, plane.form, boundaries=NO_LOOP_BOUNDARIES) + for plane, played in zip(PLANES, planes, strict=True) + ) started = process_time() table, streams = encode_streams( - planes, + packed, seeds, options=EVERY_LAYER, - boundaries=(NO_LOOP_BOUNDARIES,) * len(planes), + boundaries=(NO_LOOP_BOUNDARIES,) * len(packed), ) seconds = process_time() - started return Encoding( + header=SONG_HEADER_SIZE, phrases=len(table), dictionary=table.size, streams=tuple(len(stream) for stream in streams), seconds=seconds, lossless=all( - decode_plane(stream, table, len(plane)) == plane for plane, stream in zip(planes, streams, strict=True) + decode_plane(stream, table, plane, len(played)) == played + for plane, played, stream in zip(PLANES, planes, streams, strict=True) ), written=None, ) diff --git a/src/sampletones_tools/codec/study/layouts/planes.py b/src/sampletones_tools/codec/study/layouts/planes.py index d9449fee0..8f8f130d9 100644 --- a/src/sampletones_tools/codec/study/layouts/planes.py +++ b/src/sampletones_tools/codec/study/layouts/planes.py @@ -1,13 +1,17 @@ -from typing import List, Sequence, Tuple +from typing import Final, FrozenSet, List, Sequence, Tuple +from sampletones_core.constants.enums import TONE_CHANNELS, ChannelName from sampletones_player.compression.dictionary.phrase import Phrase from sampletones_player.compression.pitch import PitchTable -from sampletones_player.compression.planes.channel import TonePlanes +from sampletones_player.compression.planes.symbols import pack_plane from sampletones_player.specification.compression import MAX_PHRASE_LENGTH +from sampletones_player.specification.planes import PLANES, channel_indices from sampletones_tools.codec.study.corpus.song import StudySlice, StudySong from sampletones_tools.codec.study.layouts.layout import PlaneLayout from sampletones_tools.codec.study.layouts.tone import tone_planes +NO_BOUNDARIES: Final[FrozenSet[int]] = frozenset() + def layout_planes( song: StudySong, @@ -22,12 +26,12 @@ def layout_planes( Returns: Tuple[bytes, ...]: The planes; a flagged bend plane covers fewer values than the song's ticks. """ - tones = (song.planes.pulse1, song.planes.pulse2, song.planes.triangle) + tones = tuple(song.planes.of(channel) for channel in ChannelName.items() if channel in TONE_CHANNELS) planes: List[bytes] = [] for channel, notes in zip(tones, song.notes, strict=True): planes.extend(tone_planes(channel, notes, song.pitches, layout)) - planes.extend(song.planes.noise.ordered) + planes.extend(song.planes.of(ChannelName.NOISE)) return tuple(planes) @@ -51,12 +55,16 @@ def layout_seeds( """ phrases: List[Phrase] = [] for study_slice in slices: - match study_slice.planes: - case TonePlanes(): - planes: Tuple[bytes, ...] = tone_planes(study_slice.planes, study_slice.notes, pitches, layout) - case _: - planes = study_slice.planes.ordered + planes = ( + tone_planes(study_slice.planes, study_slice.notes, pitches, layout) + if study_slice.channel in TONE_CHANNELS + else study_slice.planes + ) - phrases.extend(Phrase(body=plane[:MAX_PHRASE_LENGTH]) for plane in planes if len(set(plane)) > 1) + phrases.extend( + Phrase(body=pack_plane(plane, PLANES[index].form, boundaries=NO_BOUNDARIES)[:MAX_PHRASE_LENGTH]) + for index, plane in zip(channel_indices(study_slice.channel), planes, strict=True) + if len(set(plane)) > 1 + ) return tuple(phrases) diff --git a/src/sampletones_tools/codec/study/layouts/tone.py b/src/sampletones_tools/codec/study/layouts/tone.py index 1e2b1333b..cf5849ed1 100644 --- a/src/sampletones_tools/codec/study/layouts/tone.py +++ b/src/sampletones_tools/codec/study/layouts/tone.py @@ -2,11 +2,11 @@ from sampletones_core.timers.nearest import NearestPitch from sampletones_player.compression.pitch import PitchTable -from sampletones_player.compression.planes.channel import TonePlanes from sampletones_player.compression.planes.flags import flagged_value, note_flags from sampletones_player.compression.planes.rebuild import tone_dividers from sampletones_player.registers.dividers import anchored_pitches from sampletones_player.specification.binary import unsigned_byte +from sampletones_player.specification.planes import SILENT_PITCH_INDEX from sampletones_tools.codec.study.layouts.layout import Anchor, BendForm, PlaneLayout @@ -74,12 +74,15 @@ def bend_flags( def tone_planes( - planes: TonePlanes, + planes: Tuple[bytes, ...], notes: bytes, pitches: PitchTable, layout: PlaneLayout, -) -> Tuple[bytes, bytes, bytes]: - """A tone channel's control, value and bend planes written under a layout. +) -> Tuple[bytes, ...]: + """A tone channel's planes written under a layout, the divider read across value and bend. + + A tick naming the index that stands for silence keeps it, since a layout moves where a + divider is written rather than whether the channel sounds. Args: planes: The channel's planes as the production codec separates them. @@ -88,17 +91,25 @@ def tone_planes( layout: How the divider is written across the value and the bend. Returns: - Tuple[bytes, bytes, bytes]: The control, value and bend planes; a flagged bend plane holds - only the ticks its value plane flags. + Tuple[bytes, ...]: The channel's planes as the song block writes them; a flagged bend + plane holds only the ticks its value plane flags. Raises: ValueError: If a flagged layout meets an index reaching the flag's bit. """ - anchored = layout_anchors(tone_dividers(planes, pitches.timers), notes, pitches, layout.anchor) - flags = bend_flags(anchored, layout.form) + *timbre, named, offsets = planes + resting = tuple(pitch == SILENT_PITCH_INDEX for pitch in named) + anchored = layout_anchors(tone_dividers(named, offsets, pitches.timers), notes, pitches, layout.anchor) + flags = tuple(flag and not rest for flag, rest in zip(bend_flags(anchored, layout.form), resting, strict=True)) bend = bytes(unsigned_byte(pitch.offset) for pitch, flag in zip(anchored, flags) if flag) if layout.form is BendForm.DENSE: - return planes.control, bytes(pitch.pitch for pitch in anchored), bend - - value = bytes(flagged_value(pitch.pitch, flag) for pitch, flag in zip(anchored, flags)) - return planes.control, value, bend + spelled = bytes( + SILENT_PITCH_INDEX if rest else pitch.pitch for rest, pitch in zip(resting, anchored, strict=True) + ) + return (*timbre, spelled, bend) + + value = bytes( + SILENT_PITCH_INDEX if rest else flagged_value(pitch.pitch, flag) + for rest, pitch, flag in zip(resting, anchored, flags, strict=True) + ) + return (*timbre, value, bend) diff --git a/src/sampletones_tools/codec/study/measure.py b/src/sampletones_tools/codec/study/measure.py index e116fcec4..22c674733 100644 --- a/src/sampletones_tools/codec/study/measure.py +++ b/src/sampletones_tools/codec/study/measure.py @@ -16,6 +16,7 @@ class Encoding: stated as the bytes each would take, so the two kinds sit in one report on the same terms. Attributes: + header: The bytes the song block's header takes, which a format stating more of itself grows. phrases: The phrases the dictionary holds. dictionary: The bytes the dictionary takes. streams: The bytes each plane's stream takes, in the order the song block writes them. @@ -24,6 +25,7 @@ class Encoding: written: The streams as the driver reads them, where the variant writes its grammar. """ + header: int phrases: int dictionary: int streams: Tuple[int, ...] @@ -52,7 +54,7 @@ class Measurement: @property def block(self) -> int: """The bytes the whole song block takes: header, pitch table, dictionary and streams.""" - return SONG_HEADER_SIZE + len(self.song.pitches.data) + self.dictionary + self.streams + return self.encoding.header + len(self.song.pitches.data) + self.dictionary + self.streams @property def dictionary(self) -> int: @@ -101,6 +103,7 @@ def production_encoding( Encoding: The encoding, its streams kept as written. """ return Encoding( + header=SONG_HEADER_SIZE, phrases=len(compressed.phrases), dictionary=compressed.phrases.size, streams=tuple(len(stream) for stream in compressed.streams), diff --git a/src/sampletones_tools/codec/study/sandbox/context.py b/src/sampletones_tools/codec/study/sandbox/context.py index 60051a41f..78e942084 100644 --- a/src/sampletones_tools/codec/study/sandbox/context.py +++ b/src/sampletones_tools/codec/study/sandbox/context.py @@ -4,7 +4,6 @@ from sampletones_player.compression.matches.index import PlaneIndex from sampletones_player.compression.matches.matcher import PhraseMatcher from sampletones_player.compression.parse.boundaries import Boundaries -from sampletones_player.specification.compression import INITIAL_PLANE_VALUE from sampletones_tools.codec.study.sandbox.costs import Costs @@ -14,10 +13,11 @@ class PlaneContext: Attributes: index: The plane, its steps and its runs. - matcher: Which phrases the plane plays at a tick, and for how long. - boundaries: The ticks a token starts on, read either way from every tick. + matcher: Which phrases the plane plays at a symbol, and for how long. + boundaries: The symbols a token starts on, read either way from every symbol. transposition: Whether a phrase may play at a shift. defaults: The default count of each phrase, by id, zero where the phrase has none. + seeded: The symbol the plane stands at before its first token. costs: The bytes each token takes. """ @@ -26,18 +26,19 @@ class PlaneContext: boundaries: Boundaries transposition: bool defaults: Tuple[int, ...] + seeded: int costs: Costs def value_before(self, position: int) -> int: - """The value the plane holds as ``position`` is reached, which a hold there keeps. + """The symbol the plane holds as ``position`` is reached, which a hold there keeps. - The driver seeds every plane before the first token, so the stream's first tick is - reached holding that value like any other. + The driver seeds every plane before the first token, so the stream's first symbol is + reached holding the seeded symbol like any other. Args: - position: The tick. + position: The symbol. Returns: - int: The value. + int: The symbol held. """ - return self.index.plane[position - 1] if position > 0 else INITIAL_PLANE_VALUE + return self.index.plane[position - 1] if position > 0 else self.seeded diff --git a/src/sampletones_tools/codec/study/sandbox/costs.py b/src/sampletones_tools/codec/study/sandbox/costs.py index 23d3f1703..d05045678 100644 --- a/src/sampletones_tools/codec/study/sandbox/costs.py +++ b/src/sampletones_tools/codec/study/sandbox/costs.py @@ -7,13 +7,12 @@ CHEAP_PHRASE_IDS, OPCODE_SIZE, PHRASE_COUNT_SIZE, + PHRASE_DEFAULT_SIZE, PHRASE_ESCAPE_SIZE, TRANSPOSE_SIZE, ) VALUE_SIZE: Final[int] = 1 -DEFAULT_COUNT_SIZE: Final[int] = 1 -DEFAULT_FLAG_BITS: Final[int] = 1 @dataclass(frozen=True) @@ -87,7 +86,6 @@ def dictionary(self, table: PhraseTable) -> int: phrase_escape=PHRASE_ESCAPE_SIZE, transpose=TRANSPOSE_SIZE, operands=CHEAP_PHRASE_IDS, - default_entry=0, + default_entry=PHRASE_DEFAULT_SIZE, ) SET_HOLD_BOUND: Final[int] = OPCODE_SIZE + VALUE_SIZE -DEFAULT_COUNT_OPERANDS: Final[int] = (CHEAP_PHRASE_IDS + 1) // (1 << DEFAULT_FLAG_BITS) - 1 diff --git a/src/sampletones_tools/codec/study/sandbox/encode.py b/src/sampletones_tools/codec/study/sandbox/encode.py index 182429300..3f84729d4 100644 --- a/src/sampletones_tools/codec/study/sandbox/encode.py +++ b/src/sampletones_tools/codec/study/sandbox/encode.py @@ -1,6 +1,7 @@ from time import process_time from typing import Final, Sequence, Tuple +from sampletones_player.specification.song import SONG_HEADER_SIZE from sampletones_tools.codec.study.measure import Encoding from sampletones_tools.codec.study.sandbox.defaults import modal_counts, no_defaults from sampletones_tools.codec.study.sandbox.grammar import Grammar @@ -28,11 +29,12 @@ def encode_grammar( parses = _parses(reference, grammar) seconds = process_time() - started return Encoding( + header=SONG_HEADER_SIZE, phrases=len(reference.table), dictionary=grammar.costs.dictionary(reference.table), streams=tuple(parse.size for parse in parses), seconds=seconds, - lossless=plays_back(parses, reference.table, reference.song.planes), + lossless=plays_back(parses, reference.table, reference.planes), written=None, ) diff --git a/src/sampletones_tools/codec/study/sandbox/grammar.py b/src/sampletones_tools/codec/study/sandbox/grammar.py index 1827e2924..81cb010e8 100644 --- a/src/sampletones_tools/codec/study/sandbox/grammar.py +++ b/src/sampletones_tools/codec/study/sandbox/grammar.py @@ -35,6 +35,6 @@ class Grammar: start_hold=False, wide_hold=False, set_hold=False, - default_counts=False, + default_counts=True, costs=PRODUCTION_COSTS, ) diff --git a/src/sampletones_tools/codec/study/sandbox/reference.py b/src/sampletones_tools/codec/study/sandbox/reference.py index b370ff6cf..850d5f1f7 100644 --- a/src/sampletones_tools/codec/study/sandbox/reference.py +++ b/src/sampletones_tools/codec/study/sandbox/reference.py @@ -1,7 +1,6 @@ from dataclasses import dataclass from typing import Final, FrozenSet, Sequence, Tuple -from sampletones_player.compression.absent import is_absent from sampletones_player.compression.compressed import CompressedPlanes from sampletones_player.compression.dictionary.table import PhraseTable from sampletones_player.compression.encode import STREAM_START @@ -10,15 +9,17 @@ from sampletones_player.compression.matches.matcher import PhraseMatcher from sampletones_player.compression.options import EVERY_LAYER from sampletones_player.compression.parse.boundaries import Boundaries +from sampletones_player.compression.planes.symbols import pack_plane +from sampletones_player.specification.planes import PLANES, SINGLE_TICK from sampletones_tools.codec.study.corpus.song import StudySong from sampletones_tools.codec.study.sandbox.context import PlaneContext from sampletones_tools.codec.study.sandbox.costs import Costs -from sampletones_tools.codec.study.sandbox.defaults import no_defaults from sampletones_tools.codec.study.sandbox.grammar import BASELINE_GRAMMAR, Grammar from sampletones_tools.codec.study.sandbox.parse import StudyParse, parse_plane from sampletones_tools.codec.study.sandbox.verify import verify_baseline STREAM_ENTRIES: Final[FrozenSet[int]] = frozenset({STREAM_START}) +NO_BOUNDARIES: Final[FrozenSet[int]] = frozenset() ABSENT_PARSE: Final[StudyParse] = StudyParse(tokens=(), costs=(0,)) @@ -35,14 +36,18 @@ class Reference: Attributes: song: The song. + planes: The byte series each stream was written from, in song-block order. table: The dictionary the production codec settled on. cache: What each phrase plays against each written plane, shared by every grammar's parse. + seeds: The symbol each written plane stands at before its first token. baseline: Every plane under the baseline grammar, held to the production streams. """ song: StudySong + planes: Tuple[bytes, ...] table: PhraseTable cache: MatchCache + seeds: Tuple[int, ...] baseline: Tuple[StudyParse, ...] def parses( @@ -59,7 +64,7 @@ def parses( Returns: Tuple[StudyParse, ...]: One parse per plane, in song-block order, an absent plane's empty. """ - return _parses(self.song, self.cache, self.table, grammar, defaults) + return plane_parses(self.planes, self.cache, self.table, grammar, defaults, self.seeds) def _contexts( @@ -67,6 +72,7 @@ def _contexts( table: PhraseTable, costs: Costs, defaults: Sequence[int], + seeds: Sequence[int], ) -> Tuple[PlaneContext, ...]: contexts = [] for plane, index in enumerate(cache.indices): @@ -77,6 +83,7 @@ def _contexts( boundaries=Boundaries.across(index.ticks, STREAM_ENTRIES), transposition=EVERY_LAYER.transposition, defaults=tuple(defaults), + seeded=seeds[plane], costs=costs, ) ) @@ -84,15 +91,17 @@ def _contexts( return tuple(contexts) -def _parses( - song: StudySong, +def plane_parses( + planes: Sequence[bytes], cache: MatchCache, table: PhraseTable, grammar: Grammar, defaults: Sequence[int], + seeds: Sequence[int], ) -> Tuple[StudyParse, ...]: - written = iter(parse_plane(context, grammar) for context in _contexts(cache, table, grammar.costs, defaults)) - return tuple(ABSENT_PARSE if is_absent(plane) else next(written) for plane in song.planes.planes) + contexts = _contexts(cache, table, grammar.costs, defaults, seeds) + written = iter(parse_plane(context, grammar) for context in contexts) + return tuple(next(written) if plane else ABSENT_PARSE for plane in planes) def reference( @@ -111,13 +120,23 @@ def reference( Raises: ValueError: If the baseline grammar prices a plane differently from the codec's stream. """ - cache = MatchCache(PlaneIndex.from_plane(plane) for plane in song.planes.planes if not is_absent(plane)) + planes = tuple( + b"" if plane.idles(played) else pack_plane(played, plane.form, boundaries=NO_BOUNDARIES) + for plane, played in zip(PLANES, song.planes.planes, strict=True) + ) + cache = MatchCache(PlaneIndex.from_plane(plane) for plane in planes if plane) + seeds = tuple( + plane.form.symbol(plane.seeded, SINGLE_TICK) for plane, written in zip(PLANES, planes, strict=True) if written + ) table = compressed.phrases - baseline = _parses(song, cache, table, BASELINE_GRAMMAR, no_defaults(len(table))) + carried = tuple(phrase.default for phrase in table.phrases) + baseline = plane_parses(planes, cache, table, BASELINE_GRAMMAR, carried, seeds) verify_baseline(baseline, compressed) return Reference( song=song, + planes=planes, table=table, cache=cache, + seeds=seeds, baseline=baseline, ) diff --git a/src/sampletones_tools/codec/study/sandbox/verify.py b/src/sampletones_tools/codec/study/sandbox/verify.py index 21e1a7a9c..43c3b68af 100644 --- a/src/sampletones_tools/codec/study/sandbox/verify.py +++ b/src/sampletones_tools/codec/study/sandbox/verify.py @@ -3,7 +3,6 @@ from sampletones_player.compression.compressed import CompressedPlanes from sampletones_player.compression.dictionary.table import PhraseTable from sampletones_player.compression.planes.order import PlaneOrder -from sampletones_player.compression.planes.song import SongPlanes from sampletones_tools.codec.study.sandbox.decode import play_tokens from sampletones_tools.codec.study.sandbox.parse import StudyParse @@ -36,16 +35,16 @@ def verify_baseline( def plays_back( parses: Sequence[StudyParse], table: PhraseTable, - planes: SongPlanes, + planes: Sequence[bytes], ) -> bool: """Whether every plane's tokens play back to the plane they were written from. Args: parses: Every plane's parse, in the order the song block writes them. table: The dictionary the tokens name. - planes: The planes the parses were read from. + planes: The byte series the parses were read from, in song-block order. Returns: bool: Whether the encoding is lossless. """ - return all(play_tokens(parse.tokens, table, len(plane)) == plane for parse, plane in zip(parses, planes.planes)) + return all(play_tokens(parse.tokens, table, len(plane)) == plane for parse, plane in zip(parses, planes)) diff --git a/src/sampletones_tools/codec/study/variants/sandbox.py b/src/sampletones_tools/codec/study/variants/sandbox.py index bb63a6769..6cbaadca6 100644 --- a/src/sampletones_tools/codec/study/variants/sandbox.py +++ b/src/sampletones_tools/codec/study/variants/sandbox.py @@ -4,8 +4,6 @@ from sampletones_tools.codec.study.corpus.song import StudySong from sampletones_tools.codec.study.measure import Encoder, Encoding from sampletones_tools.codec.study.sandbox.costs import ( - DEFAULT_COUNT_OPERANDS, - DEFAULT_COUNT_SIZE, PRODUCTION_COSTS, SET_HOLD_BOUND, Costs, @@ -17,7 +15,6 @@ WIDE_HOLD: Final[str] = "H1" SET_HOLD: Final[str] = "H3" -DEFAULT_COUNT: Final[str] = "H4" HOLD_EDGES: Final[str] = "H9" DRIVER_UNCHANGED: Final[str] = "driver unchanged" WIDE_HOLD_NOTE: Final[str] = ( @@ -30,17 +27,8 @@ SET_HOLD_BOUND_NOTE: Final[str] = ( "a two-byte set-hold needs an opcode of its own, so this bounds what any reallocation of the opcode space reaches" ) -DEFAULT_COUNT_NOTE: Final[str] = ( - "one operand bit says the count is the phrase's own, leaving 31 cheap ids; each table entry " - "grows by a byte, read once when a phrase is set" -) COMBINED_NOTE: Final[str] = "every change the free escape carries at once, the escape counting 31 blocks and 31 ticks" -DEFAULT_COUNT_COSTS: Final[Costs] = replace( - PRODUCTION_COSTS, - operands=DEFAULT_COUNT_OPERANDS, - default_entry=DEFAULT_COUNT_SIZE, -) SET_HOLD_BOUND_COSTS: Final[Costs] = replace( PRODUCTION_COSTS, set_hold=SET_HOLD_BOUND, @@ -108,23 +96,9 @@ class GrammarVariant(NamedTuple): note=SET_HOLD_BOUND_NOTE, grammar=replace(BASELINE_GRAMMAR, set_hold=True, costs=SET_HOLD_BOUND_COSTS), ), - GrammarVariant( - name="default-count", - hypothesis=DEFAULT_COUNT, - kind=VariantKind.FORMAT, - note=DEFAULT_COUNT_NOTE, - grammar=replace(BASELINE_GRAMMAR, default_counts=True, costs=DEFAULT_COUNT_COSTS), - ), - GrammarVariant( - name="wide-hold+default-count", - hypothesis=f"{WIDE_HOLD}+{DEFAULT_COUNT}", - kind=VariantKind.FORMAT, - note=f"{WIDE_HOLD_NOTE}; {DEFAULT_COUNT_NOTE}", - grammar=replace(BASELINE_GRAMMAR, wide_hold=True, default_counts=True, costs=DEFAULT_COUNT_COSTS), - ), GrammarVariant( name="escape-grammar", - hypothesis=f"{WIDE_HOLD}+{SET_HOLD}+{DEFAULT_COUNT}+{HOLD_EDGES}", + hypothesis=f"{WIDE_HOLD}+{SET_HOLD}+{HOLD_EDGES}", kind=VariantKind.FORMAT, note=COMBINED_NOTE, grammar=replace( @@ -132,8 +106,6 @@ class GrammarVariant(NamedTuple): start_hold=True, wide_hold=True, set_hold=True, - default_counts=True, - costs=DEFAULT_COUNT_COSTS, ), ), ) diff --git a/src/sampletones_tools/player/assembler/labels.py b/src/sampletones_tools/player/assembler/labels.py index 32f7a9c4b..4001a2c21 100644 --- a/src/sampletones_tools/player/assembler/labels.py +++ b/src/sampletones_tools/player/assembler/labels.py @@ -6,6 +6,7 @@ from sampletones_shared.exceptions import DriverBuildError LABEL_MARKER: Final[str] = "al" +LABEL_ENCODING: Final[str] = "utf-8" LABEL_FIELDS: Final[int] = 3 SYMBOL_PREFIX: Final[str] = "." @@ -29,7 +30,7 @@ def read_labels(path: Path) -> Dict[str, int]: Dict[str, int]: Each symbol's address, keyed by the symbol's own name. """ labels: Dict[str, int] = {} - for line in path.read_text().splitlines(): + for line in path.read_text(encoding=LABEL_ENCODING).splitlines(): fields = line.split() if len(fields) == LABEL_FIELDS and fields[0] == LABEL_MARKER: labels[fields[2].removeprefix(SYMBOL_PREFIX)] = int( diff --git a/src/sampletones_tools/player/assembly/include/song.inc b/src/sampletones_tools/player/assembly/include/song.inc index e5098be3e..ee020b183 100644 --- a/src/sampletones_tools/player/assembly/include/song.inc +++ b/src/sampletones_tools/player/assembly/include/song.inc @@ -1,5 +1,5 @@ WORD_SIZE = 2 -PLANE_COUNT = 11 +PLANE_COUNT = 10 STEP_WHOLE_OFFSET = 0 STEP_FRACTION_OFFSET = STEP_WHOLE_OFFSET + 1 @@ -18,6 +18,10 @@ TICK_PLAYS = 1 TICK_REPEATED = 2 PITCH_COUNT = 104 +SILENT_PITCH_INDEX = $7F +TRIANGLE_COUNTER = $80 +TRIANGLE_SOUNDING = $7F +TRIANGLE_SILENT = $00 BEND_FLAG = $80 PITCH_INDEX_MASK = BEND_FLAG - 1 @@ -29,18 +33,32 @@ TAG_PHRASE = $80 TAG_TRANSPOSED_PHRASE = $C0 OPCODE_SIZE = 1 -PHRASE_ID_ESCAPE = $3F +DEFAULT_COUNT_FLAG = $20 +PHRASE_ID_MASK = $1F +PHRASE_ID_ESCAPE = $1F PHRASE_TABLE_COUNT_SIZE = 1 PHRASE_TABLE_ENTRY_SIZE = 2 PHRASE_LENGTH_SIZE = 1 +PHRASE_DEFAULT_SIZE = 1 -PLANE_STATE_SIZE = 8 +COUNT_STEP = $10 +COUNT_SHIFT = 4 +PULSE_CONTROL_MASK = $CF +PULSE_CONTROL_FIXED = $30 +NOISE_CONTROL_MASK = $0F +NOISE_CONTROL_FIXED = $30 +NOISE_VALUE_MASK = $8F +NOISE_VALUE_FIXED = $00 + +PLANE_STATE_SIZE = 10 PLANE_SOURCE = 0 PLANE_PHRASE = 2 PLANE_PHRASE_TICKS = 4 PLANE_TOKEN_TICKS = 5 PLANE_VALUE = 6 PLANE_SHIFT = 7 +PLANE_REPEATS = 8 +PLANE_COUNT_MASK = 9 PLANE_STATE_BYTES = PLANE_COUNT * PLANE_STATE_SIZE @@ -50,8 +68,7 @@ PULSE1_BEND_PLANE = 2 * PLANE_STATE_SIZE PULSE2_CONTROL_PLANE = 3 * PLANE_STATE_SIZE PULSE2_VALUE_PLANE = 4 * PLANE_STATE_SIZE PULSE2_BEND_PLANE = 5 * PLANE_STATE_SIZE -TRIANGLE_CONTROL_PLANE = 6 * PLANE_STATE_SIZE -TRIANGLE_VALUE_PLANE = 7 * PLANE_STATE_SIZE -TRIANGLE_BEND_PLANE = 8 * PLANE_STATE_SIZE -NOISE_CONTROL_PLANE = 9 * PLANE_STATE_SIZE -NOISE_VALUE_PLANE = 10 * PLANE_STATE_SIZE +TRIANGLE_VALUE_PLANE = 6 * PLANE_STATE_SIZE +TRIANGLE_BEND_PLANE = 7 * PLANE_STATE_SIZE +NOISE_CONTROL_PLANE = 8 * PLANE_STATE_SIZE +NOISE_VALUE_PLANE = 9 * PLANE_STATE_SIZE diff --git a/src/sampletones_tools/player/assembly/source/channels.s b/src/sampletones_tools/player/assembly/source/channels.s index b3b3c0020..eb8c15856 100644 --- a/src/sampletones_tools/player/assembly/source/channels.s +++ b/src/sampletones_tools/player/assembly/source/channels.s @@ -12,6 +12,7 @@ SHADOW_UNWRITTEN = $FF ABSENT_PAGE = $00 +LOWEST_PITCH_INDEX = $00 .segment "ZEROPAGE" @@ -23,6 +24,7 @@ timer_low_table: .res 2 timer_high_table: .res 2 bend: .res 1 bend_sign: .res 1 +triangle_pitch: .res 1 plane_state: .res PLANE_STATE_BYTES timer_high_shadows: .res TRIANGLE_REGISTERS + 1 @@ -31,13 +33,21 @@ timer_high_shadows: .res TRIANGLE_REGISTERS + 1 .assert OPCODE_SIZE = 1, error, "a token's operands follow its opcode by one byte" .assert PHRASE_TABLE_ENTRY_SIZE = 2, error, "a table entry is reached by one doubling" .assert PHRASE_LENGTH_SIZE = 1, error, "a phrase body follows its length by one byte" +.assert DEFAULT_COUNT_FLAG = PHRASE_ID_MASK + 1, error, "the count flag stands above every id an opcode names" .assert >song_data <> ABSENT_PAGE, lderror, "a song loaded in page zero reads as an absent plane" .assert ABSENT_STREAM = $FFFF, error, "an absent stream is told apart by both its bytes reading $FF" .assert BEND_FLAG = $80, error, "a value's flag is read as the sign bit" .assert PITCH_COUNT <= BEND_FLAG, error, "every pitch index fits below the flag" +.assert SILENT_PITCH_INDEX >= PITCH_COUNT, error, "the index standing for silence sounds no pitch" +.assert SILENT_PITCH_INDEX < BEND_FLAG, error, "the index standing for silence bends nowhere" +.assert COUNT_STEP = 1 << COUNT_SHIFT, error, "a repeat count steps from the bit it is shifted down by" +.assert PLANE_STATE_BYTES + PLANE_PHRASE < $100, error, "a plane's phrase pointer is reached in page zero" ; Readies the tables the planes read through and points every plane at its own first token. channels_reset: + lda #LOWEST_PITCH_INDEX + sta triangle_pitch + lda #SHADOW_UNWRITTEN sta timer_high_shadows + PULSE1_REGISTERS sta timer_high_shadows + PULSE2_REGISTERS @@ -116,6 +126,7 @@ seed_planes: sta plane_state + PLANE_TOKEN_TICKS,x sta plane_state + PLANE_VALUE,x sta plane_state + PLANE_SHIFT,x + sta plane_state + PLANE_REPEATS,x txa clc @@ -123,6 +134,20 @@ seed_planes: tax cpx #PLANE_STATE_BYTES bne @next + +; States what each plane's byte holds: the bits its register leaves for a repeat count, and for +; the triangle the index its value plane stands at while the channel rests. Every other plane +; keeps the zero it was seeded with, which is what makes one symbol cover one tick. +plane_forms: + lda #(PULSE_CONTROL_MASK ^ $FF) + sta plane_state + PULSE1_CONTROL_PLANE + PLANE_COUNT_MASK + sta plane_state + PULSE2_CONTROL_PLANE + PLANE_COUNT_MASK + lda #(NOISE_CONTROL_MASK ^ $FF) + sta plane_state + NOISE_CONTROL_PLANE + PLANE_COUNT_MASK + lda #(NOISE_VALUE_MASK ^ $FF) + sta plane_state + NOISE_VALUE_PLANE + PLANE_COUNT_MASK + lda #SILENT_PITCH_INDEX + sta plane_state + TRIANGLE_VALUE_PLANE + PLANE_VALUE rts ; Advances every plane the block holds by one tick of the song. A bend plane advances only on a @@ -146,8 +171,6 @@ channels_advance: ldx #PULSE2_BEND_PLANE jsr plane_step @triangle: - ldx #TRIANGLE_CONTROL_PLANE - jsr plane_step ldx #TRIANGLE_VALUE_PLANE jsr plane_step bit plane_state + TRIANGLE_VALUE_PLANE + PLANE_VALUE @@ -176,6 +199,11 @@ plane_step: ; out behind a literal, or the value the plane already reached. A body played out holds its last ; value onward, which is what carries a note whose envelope has finished. plane_advance: + lda plane_state + PLANE_REPEATS,x + beq @symbol + dec plane_state + PLANE_REPEATS,x + rts +@symbol: lda plane_state + PLANE_TOKEN_TICKS,x bne @within jsr fetch_token @@ -195,6 +223,13 @@ plane_advance: bne @held inc plane_state + PLANE_PHRASE + 1,x @held: + lda plane_state + PLANE_VALUE,x + and plane_state + PLANE_COUNT_MASK,x + lsr + lsr + lsr + lsr + sta plane_state + PLANE_REPEATS,x rts ; Reads the token the plane at X stands on into that plane's own state. @@ -250,16 +285,20 @@ fetch_token: @plays_phrase: lda opcode - and #TOKEN_OPERAND_MASK + and #PHRASE_ID_MASK cmp #PHRASE_ID_ESCAPE bne @named lda (pointer),y iny @named: jsr set_phrase + lda opcode + and #DEFAULT_COUNT_FLAG + bne @shifts lda (pointer),y iny sta plane_state + PLANE_TOKEN_TICKS,x +@shifts: lda #$00 sta plane_state + PLANE_SHIFT,x bit opcode @@ -315,18 +354,28 @@ set_phrase: ldy #$00 lda (entry),y sta plane_state + PLANE_PHRASE_TICKS,x + iny + lda (entry),y + sta plane_state + PLANE_TOKEN_TICKS,x - inc plane_state + PLANE_PHRASE,x - bne @body + clc + lda plane_state + PLANE_PHRASE,x + adc #(PHRASE_LENGTH_SIZE + PHRASE_DEFAULT_SIZE) + sta plane_state + PLANE_PHRASE,x + bcc @body inc plane_state + PLANE_PHRASE + 1,x @body: pla tay rts -; Writes what every plane last played to the registers its channel owns. +; Writes what every plane last played to the registers its channel owns. The triangle names its +; silence in the pitch its value plane carries: the index standing above every pitch the table +; holds silences the linear counter and leaves the divider where the channel last sounded. channels_write: lda plane_state + PULSE1_CONTROL_PLANE + PLANE_VALUE + and #PULSE_CONTROL_MASK + ora #PULSE_CONTROL_FIXED sta CHANNEL_CONTROL + PULSE1_REGISTERS lda plane_state + PULSE1_BEND_PLANE + PLANE_VALUE sta bend @@ -335,6 +384,8 @@ channels_write: jsr write_timer lda plane_state + PULSE2_CONTROL_PLANE + PLANE_VALUE + and #PULSE_CONTROL_MASK + ora #PULSE_CONTROL_FIXED sta CHANNEL_CONTROL + PULSE2_REGISTERS lda plane_state + PULSE2_BEND_PLANE + PLANE_VALUE sta bend @@ -342,17 +393,29 @@ channels_write: lda plane_state + PULSE2_VALUE_PLANE + PLANE_VALUE jsr write_timer - lda plane_state + TRIANGLE_CONTROL_PLANE + PLANE_VALUE + lda plane_state + TRIANGLE_VALUE_PLANE + PLANE_VALUE + cmp #SILENT_PITCH_INDEX + beq @resting + sta triangle_pitch + lda #TRIANGLE_COUNTER | TRIANGLE_SOUNDING + bne @counter +@resting: + lda #TRIANGLE_COUNTER | TRIANGLE_SILENT +@counter: sta CHANNEL_CONTROL + TRIANGLE_REGISTERS lda plane_state + TRIANGLE_BEND_PLANE + PLANE_VALUE sta bend ldx #TRIANGLE_REGISTERS - lda plane_state + TRIANGLE_VALUE_PLANE + PLANE_VALUE + lda triangle_pitch jsr write_timer lda plane_state + NOISE_CONTROL_PLANE + PLANE_VALUE + and #NOISE_CONTROL_MASK + ora #NOISE_CONTROL_FIXED sta CHANNEL_CONTROL + NOISE_REGISTERS lda plane_state + NOISE_VALUE_PLANE + PLANE_VALUE + and #NOISE_VALUE_MASK + ora #NOISE_VALUE_FIXED sta CHANNEL_TIMER_LOW + NOISE_REGISTERS rts diff --git a/tests/data/compatibility/README.md b/tests/data/compatibility/README.md index 94a3d5067..5e5e24cbf 100644 --- a/tests/data/compatibility/README.md +++ b/tests/data/compatibility/README.md @@ -50,6 +50,26 @@ It writes one document per format at the versions that build states, and leaves already archived — replacing a file would restate history under a name that already means something. The corpus grows by one file per format per version, and never changes underneath. -A version that shipped before the command existed is backfilled once, from a worktree at its tag, -with the generator kept under `generators/`. The recipe is in +A version that shipped before the command existed is backfilled once, and the release that shipped it +is what writes the file: a document the current build produced would prove nothing about what the old +build stored. So that release's code has to run. It runs against a generator written for it, which is +why the generators live here rather than at the tag — each one postdates the release it writes for. + +The tag is checked out beside the repository rather than in it, so the work in progress and the +development environment both stay as they are: + +``` +git worktree add / +cd / && uv sync --frozen +cp /tests/data/compatibility/generators/.py . +uv run python .py --output /tests/data/compatibility +cd && git worktree remove --force / +``` + +`uv sync --frozen` builds that tag's own environment from the lockfile it shipped with, so the +document is written by the dependencies the release was built on, and the version stamped inside it is +the one the installed package reports. The output lands in this directory, which leaves the worktree +disposable as soon as the generator has run. + +Why the corpus exists, and what an upgrade step owes it, is [Data compatibility](../../../docs/development/release/compatibility.md). diff --git a/tests/integration/docs/__init__.py b/tests/integration/docs/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/tests/integration/docs/test_documentation.py b/tests/integration/docs/test_documentation.py new file mode 100644 index 000000000..9e2b3121e --- /dev/null +++ b/tests/integration/docs/test_documentation.py @@ -0,0 +1,67 @@ +import re +from pathlib import Path +from typing import Final, List, Set, Tuple + +import pytest + +from sampletones_shared.paths.source import REPOSITORY_ROOT + +ENCODING: Final[str] = "utf-8" +LINK: Final[re.Pattern[str]] = re.compile(r"\[[^\]]*\]\(([^)\s]+)\)") +HEADING: Final[re.Pattern[str]] = re.compile(r"^#{1,6}\s+(.*?)\s*$", re.MULTILINE) +EXTERNAL: Final[Tuple[str, ...]] = ("http://", "https://", "mailto:") +DOCUMENTATION: Final[Path] = REPOSITORY_ROOT / "docs" +INDEX: Final[Path] = DOCUMENTATION / "index.md" +ROOT_PAGES: Final[Tuple[str, ...]] = ("README.md", "CHANGELOG.md") + + +def anchor(heading: str) -> str: + """The fragment GitHub gives a heading: punctuation dropped, every remaining space a hyphen.""" + text = re.sub(r"[`*]", "", heading).strip().lower() + return re.sub(r"[^\w\s-]", "", text).replace(" ", "-") + + +def text(page: Path) -> str: + """The page as it is stored, which is UTF-8 whatever encoding the platform prefers.""" + return page.read_text(encoding=ENCODING) + + +def anchors(page: Path) -> Set[str]: + return {anchor(heading) for heading in HEADING.findall(text(page))} + + +def pages() -> List[Path]: + return sorted(DOCUMENTATION.rglob("*.md")) + [REPOSITORY_ROOT / name for name in ROOT_PAGES] + + +def identifier(page: Path) -> str: + return page.relative_to(REPOSITORY_ROOT).as_posix() + + +class TestEveryInternalLinkResolves: + """A link between documents names a file that exists and, where it names one, a heading it holds.""" + + @pytest.mark.parametrize("page", pages(), ids=identifier) + def test_the_page_links_only_to_files_and_headings_that_exist(self, page: Path) -> None: + for target in LINK.findall(text(page)): + if target.startswith(EXTERNAL): + continue + + path, _, fragment = target.partition("#") + destination = (page.parent / path).resolve() if path else page + assert destination.exists(), f"{identifier(page)} links to a missing file: {target}" + + if fragment and destination.suffix == ".md": + assert fragment in anchors(destination), f"{identifier(page)} links to a missing heading: {target}" + + +class TestTheIndexListsEveryDocument: + """`docs/index.md` is the map of the documentation, so a page is reachable from it.""" + + def test_every_page_is_listed(self) -> None: + listed = {(INDEX.parent / target.partition("#")[0]).resolve() for target in LINK.findall(text(INDEX))} + missing = [ + identifier(page) for page in DOCUMENTATION.rglob("*.md") if page != INDEX and page.resolve() not in listed + ] + + assert not missing, f"documents missing from docs/index.md: {missing}" diff --git a/tests/integration/nsf/test_driver_bend.py b/tests/integration/nsf/test_driver_bend.py index 44cd0b2b0..ecbebf015 100644 --- a/tests/integration/nsf/test_driver_bend.py +++ b/tests/integration/nsf/test_driver_bend.py @@ -12,6 +12,7 @@ from sampletones_player.compression.scheme import CompressionScheme from sampletones_player.song import Song from sampletones_player.specification.binary import BYTE_VALUES +from sampletones_player.specification.planes import PlaneRole, plane_index from sampletones_player.specification.registers import PULSE1_TIMER_HIGH, TIMER_HIGH_SHIFT from sampletones_tools.player.trace.trace import RegisterTrace from tests.integration.nsf.console.instructions import channel_values, timer_value @@ -220,6 +221,9 @@ def test_every_tick_sounds_the_divider_the_sequencer_renders(self, test_case: Te assert sounded_dividers(song)[: len(expected)] == expected +PULSE1_BEND: Final[int] = plane_index(ChannelName.PULSE1, PlaneRole.BEND) + + class TestABentSongComingRound: """A song returning to a tick re-enters each bend plane past the flags played before it.""" @@ -238,8 +242,8 @@ def repeating(self) -> Song: return player_song(streams, NTSC_RATE, loop_tick=self.LOOP_TICK) def test_the_loop_tick_falls_after_flagged_ticks(self, repeating: Song) -> None: - pulse1 = decode_planes(repeating.planes).pulse1 - assert pulse1.bend_position(self.LOOP_TICK) > 0 + planes = decode_planes(repeating.planes) + assert planes.positions(self.LOOP_TICK)[PULSE1_BEND] > 0 def test_the_console_writes_what_the_model_states_across_its_loops(self, repeating: Song) -> None: calls = play_calls_reaching(repeating, self.ROUNDS * repeating.ticks) diff --git a/tests/integration/nsf/test_nsf_pipeline.py b/tests/integration/nsf/test_nsf_pipeline.py index 6962b0d16..22c7e00e8 100644 --- a/tests/integration/nsf/test_nsf_pipeline.py +++ b/tests/integration/nsf/test_nsf_pipeline.py @@ -14,12 +14,12 @@ from sampletones_player.nsf.song import song_to_bytes from sampletones_player.song import Song from sampletones_player.specification.binary import WORD_SIZE -from sampletones_player.specification.compression import PLANE_COUNT from sampletones_player.specification.nsf import ( HEADER_SIZE, NSF_MAGIC, PROGRAM_SIZE, ) +from sampletones_player.specification.planes import PLANE_COUNT from sampletones_player.specification.song import ( ABSENT_STREAM, LOOP_TICK_OFFSET, diff --git a/tests/integration/reconstruction/test_stems_editing.py b/tests/integration/reconstruction/test_stems_editing.py index cf58f93e3..352449c0e 100644 --- a/tests/integration/reconstruction/test_stems_editing.py +++ b/tests/integration/reconstruction/test_stems_editing.py @@ -7,6 +7,7 @@ from sampletones_core.constants.algorithm import AUTHORED_STEM_ID, RESTING_STEM_ID from sampletones_core.constants.enums import ChannelName from sampletones_core.exporters import playing_channels +from sampletones_core.formats.famitracker.footprint import reconstruction_footprints from sampletones_core.instructions import InstructionUnion, TriangleInstruction from sampletones_core.reconstructions import Reconstruction, Reconstructor from sampletones_core.reconstructions.reconstruction.stems.filter import filter_approximations @@ -462,11 +463,7 @@ def _selection(channel_name: ChannelName, stem_ids: AbstractSet[int]) -> StemSel ) def test_every_sounding_channel_reads_the_document_itself(self, reconstruction: Reconstruction) -> None: - """A reader hearing everything reads each sounding channel as the document writes it. - - A channel whose every frame rests reads as standing by instead, since the reading ends - where a channel last sounds. - """ + """A reader hearing everything reads each sounding channel as the document writes it.""" whole = reconstruction.export() sounding = {name: features for name, features in whole.items() if features.has_frames} assert sounding @@ -516,6 +513,45 @@ def test_the_channels_beside_it_read_as_they_stand(self, reconstruction: Reconst assert all(heard[name] == whole[name] for name in ChannelName.items() if name != channel_name) +class TestAChannelWrittenDownToNothing: + """A channel whose every frame rests keeps them, so every reader of it counts one channel.""" + + @staticmethod + def _silenced(reconstruction: Reconstruction, channel_name: ChannelName) -> Reconstruction: + """The document with one channel written down to rests, frame for frame.""" + stream = [type(instruction).null_instruction() for instruction in reconstruction.instructions[channel_name]] + return _edited(reconstruction, channel_name, stream) + + def test_the_channel_keeps_the_frames_it_describes(self, reconstruction: Reconstruction) -> None: + channel_name = _contested_channel(reconstruction) + frames = len(reconstruction.instructions[channel_name]) + + edited = self._silenced(reconstruction, channel_name) + + assert len(edited.instructions[channel_name]) == frames + assert not any(instruction.on for instruction in edited.instructions[channel_name]) + + def test_every_reader_counts_it_alike(self, reconstruction: Reconstruction) -> None: + """The panel, the record, the footprint and the export answer for one and the same channel.""" + channel_name = _contested_channel(reconstruction) + edited = self._silenced(reconstruction, channel_name) + everywhere = StemSelection.everywhere(EVERY_STEM, ChannelName.items()) + + assert channel_name in playing_channels(edited.export_heard(everywhere)) + assert channel_name in edited.playing_channels + assert channel_name in playing_channels(edited.export()) + assert channel_name in reconstruction_footprints(edited) + + def test_the_panel_measures_the_frames_the_export_writes(self, reconstruction: Reconstruction) -> None: + channel_name = _contested_channel(reconstruction) + edited = self._silenced(reconstruction, channel_name) + everywhere = StemSelection.everywhere(EVERY_STEM, ChannelName.items()) + + heard = edited.export_heard(everywhere)[channel_name] + + assert heard.frame_count == edited.export()[channel_name].frame_count + + class TestTheDocumentThroughAFile: def test_an_edited_document_round_trips_whole(self, reconstruction: Reconstruction, tmp_path: Path) -> None: channel_name = _contested_channel(reconstruction) diff --git a/tests/integration/reconstruction/test_stems_reconstruction.py b/tests/integration/reconstruction/test_stems_reconstruction.py index e8727cf04..358958cb9 100644 --- a/tests/integration/reconstruction/test_stems_reconstruction.py +++ b/tests/integration/reconstruction/test_stems_reconstruction.py @@ -7,7 +7,7 @@ from sampletones_application.logic.reconstruction.data import ReconstructionData from sampletones_core.audio import mix, write_wave from sampletones_core.configs import Config -from sampletones_core.constants.algorithm import RESTING_STEM_ID, UNIT_DRIVE +from sampletones_core.constants.algorithm import MAX_DRIVE, MIN_DRIVE, RESTING_STEM_ID, UNIT_DRIVE from sampletones_core.constants.enums import ( DEFAULT_CHANNELS, ChannelName, @@ -41,6 +41,9 @@ _DISJOINT_TONES: Final[Tuple[float, ...]] = (220.0, 440.0, 880.0) _DISJOINT_AMPLITUDE: Final[float] = 0.5 _LOUD_DRIVE: Final[float] = 2.0 +_SWEPT_DRIVES: Final[Tuple[float, ...]] = (MIN_DRIVE, UNIT_DRIVE, _LOUD_DRIVE, MAX_DRIVE) +_COMPETING_TONES: Final[Tuple[float, ...]] = (440.0, 659.0) +_COMPETING_AMPLITUDES: Final[Tuple[float, ...]] = (0.5, 0.4) def _classic_stems(config: Config, *, channel_cap: int) -> StemsConfig: @@ -67,6 +70,38 @@ def _stems_config(tone_drive: float = UNIT_DRIVE) -> StemsConfig: ) +def _competing_stems(drive: float) -> StemsConfig: + """Two recordings on one level, each reaching for the first pulse, the first pushed to ``drive``.""" + held = [ChannelName.PULSE1] + return StemsConfig( + entries=[ + StemEntry(id=0, settings=StemSettings.covering(held).with_channel_cap(1).with_drive(held[0], drive)), + StemEntry(id=1, settings=StemSettings.covering(held).with_channel_cap(1)), + ], + hierarchy=StemsHierarchy(levels=[[0, 1]], mode=HierarchyMode.STRICT), + ) + + +def _competing_recordings(tmp_path: Path, config: Config) -> Tuple[Path, Path]: + """Two steady tones of one length, written as the recordings that reach for one channel.""" + sample_rate = config.library.sample_rate + count = int(sample_rate * _DURATION_SECONDS) + time = np.arange(count) / sample_rate + + paths = [] + for index, (frequency, amplitude) in enumerate(zip(_COMPETING_TONES, _COMPETING_AMPLITUDES)): + path = tmp_path / f"tone_{index}.wav" + write_wave(path, sample_rate, amplitude * np.sin(2 * np.pi * frequency * time)) + paths.append(path) + + return paths[0], paths[1] + + +def _energy(samples: np.ndarray) -> float: + """How much sound a rendered channel carries, which is what a drive lifts.""" + return float(np.sum(np.square(np.asarray(samples, dtype=np.float64)))) + + def _disjoint_recordings(tmp_path: Path, config: Config) -> Tuple[Path, Path, int]: """A steady tone and a noise burst of one length, written as two recordings.""" sample_rate = config.library.sample_rate @@ -116,8 +151,8 @@ def test_assigns_disjoint_stems_to_their_channels(self, tmp_path: Path) -> None: assert len(reconstruction.instructions[ChannelName.PULSE1]) == frame_count assert len(reconstruction.instructions[ChannelName.NOISE]) == frame_count - def test_each_channel_is_recorded_at_the_drive_its_stem_gives_it(self, tmp_path: Path) -> None: - """A channel plays the level its own stem asks for, whatever the stem beside it asks.""" + def test_a_driven_channel_sounds_exactly_what_its_instructions_render(self, tmp_path: Path) -> None: + """A drive settles which instruction a frame records, and the frame then sounds that instruction.""" config = Config() library = build_mini_library(config) reconstructor = Reconstructor(config, frozenset(DEFAULT_CHANNELS), library=library) @@ -132,7 +167,7 @@ def test_each_channel_is_recorded_at_the_drive_its_stem_gives_it(self, tmp_path: rendered = render_channels(reconstruction.instructions, config) np.testing.assert_allclose( reconstruction.approximations[ChannelName.PULSE1], - rendered[ChannelName.PULSE1] * _LOUD_DRIVE, + rendered[ChannelName.PULSE1], atol=_MIX_TOLERANCE, ) np.testing.assert_allclose( @@ -141,6 +176,24 @@ def test_each_channel_is_recorded_at_the_drive_its_stem_gives_it(self, tmp_path: atol=_MIX_TOLERANCE, ) + def test_a_driven_recording_is_recorded_louder_and_leaves_the_one_beside_it(self, tmp_path: Path) -> None: + """A drive reaches for louder instructions on the channel its own recording holds, and there alone.""" + config = Config() + library = build_mini_library(config) + reconstructor = Reconstructor(config, frozenset(DEFAULT_CHANNELS), library=library) + tone_path, noise_path, _ = _disjoint_recordings(tmp_path, config) + + standing = reconstructor.reconstruct([tone_path, noise_path], _stems_config()) + driven = reconstructor.reconstruct([tone_path, noise_path], _stems_config(_LOUD_DRIVE)) + + assert standing is not None + assert driven is not None + assert _energy(driven.approximations[ChannelName.PULSE1]) > _energy(standing.approximations[ChannelName.PULSE1]) + np.testing.assert_array_equal( + driven.approximations[ChannelName.NOISE], + standing.approximations[ChannelName.NOISE], + ) + def test_requires_one_path_per_entry(self, tmp_path: Path) -> None: config = Config() library = build_mini_library(config) @@ -153,6 +206,47 @@ def test_requires_one_path_per_entry(self, tmp_path: Path) -> None: ) +class TestADriveLeavesTheStemsCompeting: + """Two recordings reaching for one channel keep it where it stood, whatever drive one is given. + + A drive settles what a channel plays, so it reaches the instructions the frames record and + leaves the ownership beneath them standing. This is the contract ``docs/concepts/stems.md`` + states of a drive, read over a whole conversion. + """ + + def test_the_channel_holds_its_recording_across_the_whole_range(self, tmp_path: Path) -> None: + config = Config() + library = build_mini_library(config) + reconstructor = Reconstructor(config, frozenset(DEFAULT_CHANNELS), library=library) + paths = list(_competing_recordings(tmp_path, config)) + + owners: Dict[float, List[int]] = {} + for drive in _SWEPT_DRIVES: + reconstruction = reconstructor.reconstruct(paths, _competing_stems(drive)) + assert reconstruction is not None + owners[drive] = list(reconstruction.stems_data.assignments_by_channel[ChannelName.PULSE1]) + + standing = owners[UNIT_DRIVE] + assert set(standing) == {0} + assert owners == {drive: standing for drive in _SWEPT_DRIVES} + + def test_the_drive_lifts_the_channel_it_holds_up_to_what_the_channel_reaches(self, tmp_path: Path) -> None: + """A rising drive reaches for louder instructions and settles at the loudest the channel holds.""" + config = Config() + library = build_mini_library(config) + reconstructor = Reconstructor(config, frozenset(DEFAULT_CHANNELS), library=library) + paths = list(_competing_recordings(tmp_path, config)) + + energies: Dict[float, float] = {} + for drive in _SWEPT_DRIVES: + reconstruction = reconstructor.reconstruct(paths, _competing_stems(drive)) + assert reconstruction is not None + energies[drive] = _energy(reconstruction.approximations[ChannelName.PULSE1]) + + assert energies[MIN_DRIVE] < energies[UNIT_DRIVE] < energies[_LOUD_DRIVE] + assert energies[MAX_DRIVE] == pytest.approx(energies[_LOUD_DRIVE]) + + class TestThreeStemHierarchy: """The shared three-stem example: a (pulse 1, triangle, noise) and b (pulse 2, triangle) pick on the first hierarchy level, c (pulse 1, noise) on the second.""" diff --git a/tests/integration/sampletones_application/services/test_regeneration.py b/tests/integration/sampletones_application/services/test_regeneration.py index f5ca11bee..3d702dff0 100644 --- a/tests/integration/sampletones_application/services/test_regeneration.py +++ b/tests/integration/sampletones_application/services/test_regeneration.py @@ -5,10 +5,13 @@ import numpy as np import pytest -from sampletones_application.logic.reconstruction.feature import FeatureData +from sampletones_application.logic.reconstruction.envelopes import heard_envelopes from sampletones_application.services.regeneration.service import RegenerationService from sampletones_application.services.result import ServiceError, ServiceSuccess from sampletones_application.utils.callbacks.queue import CallbackQueue +from sampletones_application.view_model.reconstruction.envelopes import ( + ChannelEnvelopesViewModel, +) from sampletones_core.constants.enums import ChannelName, FeatureKey from sampletones_core.exporters import Features from sampletones_core.features.envelope import Envelope @@ -17,9 +20,9 @@ from tests.suite.stems import everything_heard -def _heard_features(reconstruction: Reconstruction) -> FeatureData: +def _heard_features(reconstruction: Reconstruction) -> ChannelEnvelopesViewModel: """The envelopes of the whole document, which is what a fresh reader hears.""" - return FeatureData.heard(reconstruction, everything_heard(reconstruction)) + return heard_envelopes(reconstruction, everything_heard(reconstruction)) _real_queue_add = CallbackQueue.add diff --git a/tests/integration/tooling/test_codec_study.py b/tests/integration/tooling/test_codec_study.py index add93f6cf..7f8802b5b 100644 --- a/tests/integration/tooling/test_codec_study.py +++ b/tests/integration/tooling/test_codec_study.py @@ -16,6 +16,7 @@ from sampletones_tools.codec.study.session import run_study VARIANT: Final[str] = "wide-hold" +BASELINE_VARIANT: Final[str] = "baseline" LENGTHEN_SECONDS: Final[int] = 2 @@ -56,6 +57,29 @@ def test_a_run_writes_its_tables_and_the_manifest_that_repeats_it(self, plan: St start = markdown.index(table[0]) assert markdown[start : start + len(table)] == table + def test_a_run_reads_every_encoding_back_as_the_song_it_was_written_from( + self, + integration_project: Project, + tmp_path: Path, + ) -> None: + """A run refuses a variant whose streams do not play back, so completing is the assertion.""" + project = tmp_path / "packed.stp" + ProjectContainer.save(integration_project, project) + plan = plan_study( + StudyManifest( + projects=(StudySource.at(project),), + reconstructions=(), + lengthen_seconds=LENGTHEN_SECONDS, + variants=(BASELINE_VARIANT,), + ) + ) + + directory = run_study(plan, tmp_path / "run") + + header, *measured = _read_csv(directory / REPORT_CSV) + variant = header.index("variant") + assert {row[variant] for row in measured} >= {BASELINE_VARIANT} + def test_a_source_that_fails_to_read_leaves_the_output_untouched(self, tmp_path: Path) -> None: broken = tmp_path / "broken.stp" broken.write_bytes(b"") diff --git a/tests/suite/conversion.py b/tests/suite/conversion.py index 1be862c48..635c4c337 100644 --- a/tests/suite/conversion.py +++ b/tests/suite/conversion.py @@ -17,7 +17,7 @@ FAKE_FRAMES: Final[int] = 40 HALFWAY: Final[int] = FAKE_FRAMES // 2 COUNTED_STAGES: Final[FrozenSet[ReconstructionStage]] = frozenset( - {ReconstructionStage.MATCHING, ReconstructionStage.RENDERING} + {ReconstructionStage.MATCHING, ReconstructionStage.GATHERING} ) diff --git a/tests/suite/player.py b/tests/suite/player.py index 1e9b997aa..c3a27cb4b 100644 --- a/tests/suite/player.py +++ b/tests/suite/player.py @@ -22,7 +22,6 @@ from sampletones_player.compression.encode import emit, encode_planes from sampletones_player.compression.options import EVERY_LAYER from sampletones_player.compression.pitch import PITCH_COUNT, PitchTable -from sampletones_player.compression.planes.channel import TonePlanes from sampletones_player.compression.planes.flags import flagged_value from sampletones_player.compression.planes.order import PlaneOrder from sampletones_player.compression.planes.separate import planes_from_streams @@ -37,7 +36,13 @@ from sampletones_player.specification.binary import unsigned_byte from sampletones_player.specification.compression import ( MAX_LITERAL_BYTES, +) +from sampletones_player.specification.planes import ( PLANE_COUNT, + PLANES, + Plane, + PlaneRole, + plane_index, ) from sampletones_player.specification.registers import ( DUTY_CYCLE_SHIFT, @@ -85,14 +90,22 @@ def pulse_tick( ) +RESTING_PITCH: Final[int] = 0 + + def triangle_tick(sounding: bool, timer: int) -> TriangleRegisters: - """A triangle channel's registers for one tick, spelled the way the encoder spells them.""" + """A triangle channel's registers for one tick, spelled the way the encoder spells them. + + A resting tick states no pitch, so it carries the divider the channel would be holding: the + lowest the table reaches, which is where the encoder leaves a channel that has yet to sound. + """ reload_value = TRIANGLE_SOUNDING_RELOAD if sounding else TRIANGLE_SILENT_RELOAD + held = timer if sounding else PLAYER_PITCHES.timers[RESTING_PITCH] return TriangleRegisters( linear_counter=TRIANGLE_COUNTER_CONTROL | reload_value, - timer_low=timer & MAX_REGISTER_VALUE, - timer_high=timer >> TIMER_HIGH_SHIFT, - anchor=nearest_anchor(timer), + timer_low=held & MAX_REGISTER_VALUE, + timer_high=held >> TIMER_HIGH_SHIFT, + anchor=nearest_anchor(held), ) @@ -170,14 +183,10 @@ def bent_song( sounding = pulse_tick(PLAYER_FULL_VOLUME, 0, PLAYER_PITCHES.timers[pitch_index]) planes = planes_from_streams(resting_streams((sounding,) * len(bends)), PLAYER_PITCHES) bent = SongPlanes( - pulse1=TonePlanes( - control=planes.pulse1.control, - value=bytes(flagged_value(pitch_index, bend != 0) for bend in bends), - bend=bytes(unsigned_byte(bend) for bend in bends if bend), - ), - pulse2=planes.pulse2, - triangle=planes.triangle, - noise=planes.noise, + planes=planes.planes._replace( + pulse1_value=bytes(flagged_value(pitch_index, bend != 0) for bend in bends), + pulse1_bend=bytes(unsigned_byte(bend) for bend in bends if bend), + ) ) return Song( planes=encode_planes(bent, (), options=EVERY_LAYER, boundaries=frozenset()), @@ -210,6 +219,36 @@ def silent_pulse() -> PulseInstruction: PLAYER_VARIED_SEED: Final[int] = 7 +def playable(plane: Plane, values: bytes) -> bytes: + """Arbitrary bytes read as values the plane can play, through its own form.""" + return bytes(plane.form.value(value) for value in values) + + +def sounding_planes( + control: bytes, + value: bytes, + bend: bytes, +) -> SongPlanes: + """A song's planes, the first pulse channel sounding what it is given and the rest resting. + + Every other plane stands at the value its own register fixes, which is where the driver seeds + it and what makes it absent. + """ + resting = [b"" if plane.spans_flagged_ticks else bytes((plane.seeded,)) * len(control) for plane in PLANES] + for role, played in zip((PlaneRole.CONTROL, PlaneRole.VALUE, PlaneRole.BEND), (control, value, bend)): + resting[plane_index(ChannelName.PULSE1, role)] = played + + return SongPlanes(planes=PlaneOrder.across(resting)) + + +STREAM_START: Final[int] = 0 + + +def every_plane_spelling(stream: bytes) -> PlaneOrder: + """One stream standing for every plane the song block writes.""" + return PlaneOrder.across((stream,) * PLANE_COUNT) + + def spelled_song(ticks: int, nes_frequency: int) -> Song: """A song whose every plane spells its values out, which is the most room a song can take. @@ -227,8 +266,9 @@ def spelled_song(ticks: int, nes_frequency: int) -> Song: return Song( planes=CompressedPlanes( phrases=PhraseTable(phrases=()), - streams=PlaneOrder.across((stream,) * PLANE_COUNT), + streams=every_plane_spelling(stream), ticks=ticks, + loop_entries=(STREAM_START,) * PLANE_COUNT, ), pitches=PLAYER_PITCHES, schedule=PlaySchedule.from_parameters(nes_frequency), diff --git a/tests/suite/study.py b/tests/suite/study.py index ef2161faf..08dc0f3c0 100644 --- a/tests/suite/study.py +++ b/tests/suite/study.py @@ -1,7 +1,8 @@ from typing import Final, Tuple -from sampletones_player.compression.planes.channel import ChannelPlanes, TonePlanes +from sampletones_player.compression.planes.order import PlaneOrder from sampletones_player.compression.planes.song import SongPlanes +from sampletones_player.specification.planes import PLANES from sampletones_shared.constants.music import LIMIT_MIN_PITCH from sampletones_tools.codec.study.corpus.notes import TONE_ORDER from sampletones_tools.codec.study.corpus.song import StudySlice @@ -16,10 +17,4 @@ def lowest_notes(ticks: int) -> Tuple[bytes, ...]: def resting_planes(ticks: int) -> SongPlanes: """A song's planes holding zero throughout, every tone channel bending nowhere.""" - tone = TonePlanes(control=bytes(ticks), value=bytes(ticks), bend=b"") - return SongPlanes( - pulse1=tone, - pulse2=tone, - triangle=tone, - noise=ChannelPlanes(control=bytes(ticks), value=bytes(ticks)), - ) + return SongPlanes(planes=PlaneOrder.across(b"" if plane.spans_flagged_ticks else bytes(ticks) for plane in PLANES)) diff --git a/tests/unit/sampletones_application/layout/test_about_dialog.py b/tests/unit/sampletones_application/layout/test_about_dialog.py index 99296ccaa..53c05f2ae 100644 --- a/tests/unit/sampletones_application/layout/test_about_dialog.py +++ b/tests/unit/sampletones_application/layout/test_about_dialog.py @@ -1,7 +1,13 @@ from sampletones_application.layout.general.dialogs.about import AboutDialogLayout +from sampletones_application.layout.primitives import DialogGeometry class TestTheRoomTheTextTakes: def test_the_text_wraps_in_what_the_mark_leaves(self) -> None: - layout = AboutDialogLayout(width=480, height=210, logo=72, padding=40) + layout = AboutDialogLayout( + window=DialogGeometry(width=480, height=210), + logo=72, + padding=40, + ) + assert layout.text_wrap == 368 diff --git a/tests/unit/sampletones_application/layout/test_primitives.py b/tests/unit/sampletones_application/layout/test_primitives.py new file mode 100644 index 000000000..c4695a0b2 --- /dev/null +++ b/tests/unit/sampletones_application/layout/test_primitives.py @@ -0,0 +1,36 @@ +from typing import Final + +from sampletones_application.layout.primitives import DEARPYGUI_MAXIMUM_WINDOW_SIZE, DialogGeometry + +STATED_HEIGHT: Final[int] = 210 +STATED_WIDTH: Final[int] = 480 + + +class TestTheSizeADialogIsHeldTo: + """A dialog reads at the width it states and grows down to hold what it holds. + + A window free to widen to its content, holding content that measures itself against the + window's width, hands each a little more every frame until the screen stops it. + """ + + def test_the_width_a_dialog_opens_at_is_the_width_it_stops_at(self) -> None: + geometry = DialogGeometry(width=STATED_WIDTH, height=STATED_HEIGHT) + + assert geometry.minimum_size[0] == geometry.maximum_size[0] == STATED_WIDTH + + def test_a_dialog_stating_no_height_holds_its_width_all_the_same(self) -> None: + geometry = DialogGeometry(width=STATED_WIDTH) + + assert geometry.minimum_size[0] == geometry.maximum_size[0] == STATED_WIDTH + + def test_the_height_a_dialog_opens_at_is_the_least_it_takes(self) -> None: + geometry = DialogGeometry(width=STATED_WIDTH, height=STATED_HEIGHT) + + assert geometry.minimum_size[1] == STATED_HEIGHT + assert geometry.maximum_size[1] == DEARPYGUI_MAXIMUM_WINDOW_SIZE + + def test_a_dialog_stating_no_height_starts_from_its_content(self) -> None: + geometry = DialogGeometry(width=STATED_WIDTH) + + assert geometry.minimum_size[1] == 0 + assert geometry.maximum_size[1] == DEARPYGUI_MAXIMUM_WINDOW_SIZE diff --git a/tests/unit/sampletones_application/logic/main/converter/texts.py b/tests/unit/sampletones_application/logic/main/converter/texts.py index cc59c2017..5a2327831 100644 --- a/tests/unit/sampletones_application/logic/main/converter/texts.py +++ b/tests/unit/sampletones_application/logic/main/converter/texts.py @@ -15,7 +15,8 @@ "main.converter.message.stage_loading": "reading", "main.converter.message.stage_matching": "matching", "main.converter.message.stage_decoding": "decoding", - "main.converter.message.stage_rendering": "rendering", + "main.converter.message.stage_gathering": "gathering", + "global.dialog.label.unknown_duration": "?", "global.dialog.template.time_estimation": "", } diff --git a/tests/unit/sampletones_application/logic/reconstruction/test_editor.py b/tests/unit/sampletones_application/logic/reconstruction/test_editor.py index c2fb0ccc8..b9b7b2490 100644 --- a/tests/unit/sampletones_application/logic/reconstruction/test_editor.py +++ b/tests/unit/sampletones_application/logic/reconstruction/test_editor.py @@ -6,9 +6,12 @@ from sampletones_application.logic.history.manager import HistoryManager from sampletones_application.logic.project.controller import ProjectController from sampletones_application.logic.project.manager import ProjectManager -from sampletones_application.logic.reconstruction.editing import InstrumentEdit, ReconstructionEdit +from sampletones_application.logic.reconstruction.editing import InstrumentEdit from sampletones_application.logic.reconstruction.editor import InstrumentEditor from sampletones_application.logic.reconstruction.manager import ReconstructionManager +from sampletones_application.view_model.reconstruction.envelopes import ( + ChannelEnvelopesViewModel, +) from sampletones_core.constants.enums import ChannelName, FeatureKey from sampletones_core.exporters import Features from sampletones_core.features.envelope import Envelope @@ -69,11 +72,14 @@ def test_a_loaded_reconstruction_answers_with_its_channels( editor: InstrumentEditor, reconstruction_manager: MagicMock, ) -> None: - reconstruction_manager.current_features = MagicMock(channels={ChannelName.PULSE1: _features()}) + reconstruction_manager.current_features = ChannelEnvelopesViewModel( + channels={ChannelName.PULSE1: _features()}, + ownership={}, + ) edit = editor.edited_instrument() - assert isinstance(edit, ReconstructionEdit) + assert isinstance(edit, ChannelEnvelopesViewModel) assert list(edit.channels) == [ChannelName.PULSE1] def test_an_instrument_answers_with_what_it_states( @@ -114,11 +120,14 @@ def test_letting_go_of_an_instrument_hands_the_tab_back( ) -> None: instrument = controller.add_instrument(new_instrument("lead")) editor.edit_instrument(instrument.id) - reconstruction_manager.current_features = MagicMock(channels={ChannelName.PULSE1: _features()}) + reconstruction_manager.current_features = ChannelEnvelopesViewModel( + channels={ChannelName.PULSE1: _features()}, + ownership={}, + ) editor.release_instrument() - assert isinstance(editor.edited_instrument(), ReconstructionEdit) + assert isinstance(editor.edited_instrument(), ChannelEnvelopesViewModel) def test_an_instrument_removed_from_the_project_leaves_the_tab_holding_nothing( self, diff --git a/tests/unit/sampletones_application/logic/reconstruction/test_envelopes.py b/tests/unit/sampletones_application/logic/reconstruction/test_envelopes.py new file mode 100644 index 000000000..2c9c0ba39 --- /dev/null +++ b/tests/unit/sampletones_application/logic/reconstruction/test_envelopes.py @@ -0,0 +1,193 @@ +from __future__ import annotations + +from typing import Callable + +import pytest + +from sampletones_application.logic.reconstruction.envelopes import heard_envelopes +from sampletones_application.view_model.reconstruction.envelopes import ( + ChannelEnvelopesViewModel, +) +from sampletones_core.constants.algorithm import RESTING_STEM_ID +from sampletones_core.constants.enums import ChannelName +from sampletones_core.exporters import Features +from sampletones_core.reconstructions import Reconstruction +from sampletones_core.reconstructions.reconstruction.stems.channel_assignment import ChannelAssignment +from sampletones_core.reconstructions.reconstruction.stems.data import StemsData +from sampletones_core.reconstructions.reconstruction.stems.selection import StemSelection +from sampletones_core.reconstructions.reconstruction.stems.source import StemSource +from sampletones_core.reconstructions.reconstructor.stems.configs.config import StemsConfig +from sampletones_core.reconstructions.reconstructor.stems.configs.entry import StemEntry +from sampletones_core.reconstructions.reconstructor.stems.configs.hierarchy import StemsHierarchy +from sampletones_core.reconstructions.reconstructor.stems.configs.settings import StemSettings +from tests.suite.stems import STEM_A_ID, STEM_B_ID + + +@pytest.fixture +def reconstruction( + reconstruction_factory: Callable[[], Reconstruction], +) -> Reconstruction: + return reconstruction_factory() + + +@pytest.fixture +def everything_heard(reconstruction: Reconstruction) -> StemSelection: + """The reader listening to every recording on every channel, as a fresh document reads.""" + return StemSelection.everywhere( + frozenset(reconstruction.stems_data.config.entries_by_id), + ChannelName.items(), + ) + + +@pytest.fixture +def envelopes(reconstruction: Reconstruction, everything_heard: StemSelection) -> ChannelEnvelopesViewModel: + return heard_envelopes(reconstruction, everything_heard) + + +@pytest.fixture +def stems_reconstruction(reconstruction: Reconstruction) -> Reconstruction: + """The same document read as two recordings, which is what gives a stretch something to name.""" + entries = [ + StemEntry(id=STEM_A_ID, settings=StemSettings.covering(list(reconstruction.playing_channels))), + StemEntry(id=STEM_B_ID, settings=StemSettings.covering(list(reconstruction.playing_channels))), + ] + assignments = [ + ChannelAssignment( + channel_name=channel_name, + stem_ids=[ + STEM_A_ID if instruction.on and frame % 2 == 0 else STEM_B_ID if instruction.on else RESTING_STEM_ID + for frame, instruction in enumerate(reconstruction.instructions[channel_name]) + ], + ) + for channel_name in reconstruction.playing_channels + ] + reconstruction.stems_data = StemsData( + config=StemsConfig(entries=entries, hierarchy=StemsHierarchy(levels=[[STEM_A_ID], [STEM_B_ID]])), + sources=[ + StemSource(stem_id=STEM_A_ID, name="first", path=None), + StemSource(stem_id=STEM_B_ID, name="second", path=None), + ], + assignments=assignments, + ) + return reconstruction + + +@pytest.fixture +def everything_heard_of(stems_reconstruction: Reconstruction) -> StemSelection: + return StemSelection.everywhere( + frozenset(stems_reconstruction.stems_data.config.entries_by_id), + ChannelName.items(), + ) + + +class TestTheEnvelopesOfWhatIsHeard: + def test_every_generator_answers_with_an_entry( + self, + envelopes: ChannelEnvelopesViewModel, + ) -> None: + assert set(envelopes.channels.keys()) == set(ChannelName.items()) + + def test_a_channel_standing_by_carries_empty_envelopes( + self, + reconstruction: Reconstruction, + envelopes: ChannelEnvelopesViewModel, + ) -> None: + """A channel the reconstruction leaves silent is loaded describing no frame.""" + standing_by = set(ChannelName.items()) - set(reconstruction.playing_channels) + assert standing_by + assert all(not envelopes[channel_name].has_frames for channel_name in standing_by) + + def test_the_envelopes_state_the_pitch_they_are_measured_against( + self, + envelopes: ChannelEnvelopesViewModel, + ) -> None: + for features in envelopes.channels.values(): + assert features.initial_pitch is not None + + def test_hearing_every_recording_reads_the_document_itself( + self, + reconstruction: Reconstruction, + envelopes: ChannelEnvelopesViewModel, + ) -> None: + """Each channel that sounds is read as the document writes it. + + A channel written down to rests alone stays in play, since every reader hears a rest. + """ + whole = reconstruction.export() + sounding = {name: features for name, features in whole.items() if features.has_frames} + assert sounding + + assert {name: envelopes[name] for name in sounding} == sounding + + def test_a_channel_no_recording_is_heard_on_describes_no_frame( + self, + reconstruction: Reconstruction, + ) -> None: + playing = next(iter(reconstruction.playing_channels)) + + envelopes = heard_envelopes(reconstruction, StemSelection(channels={})) + + assert not envelopes[playing].has_frames + + +class TestWhatTheEnvelopesAnswer: + @pytest.mark.parametrize("channel_name", ChannelName.items(), ids=lambda name: name.value) + def test_every_channel_answers_with_its_features( + self, + envelopes: ChannelEnvelopesViewModel, + channel_name: ChannelName, + ) -> None: + assert isinstance(envelopes[channel_name], Features) + + +class TestTheRecordingsBehindWhatIsDrawn: + """The stretches read the same frames the envelopes do, so a bar and its owner line up.""" + + def test_a_document_answering_to_one_recording_tells_none_apart( + self, + envelopes: ChannelEnvelopesViewModel, + ) -> None: + assert envelopes.ownership == {} + + def test_every_sounding_channel_carries_its_stretches( + self, + stems_reconstruction: Reconstruction, + everything_heard_of: StemSelection, + ) -> None: + envelopes = heard_envelopes(stems_reconstruction, everything_heard_of) + + sounding = {name for name, features in envelopes.channels.items() if features.has_frames} + assert sounding + assert set(envelopes.ownership) == sounding + + def test_a_channel_states_one_stretch_per_owner_it_changes_to( + self, + stems_reconstruction: Reconstruction, + everything_heard_of: StemSelection, + ) -> None: + envelopes = heard_envelopes(stems_reconstruction, everything_heard_of) + channel_name = next(iter(envelopes.ownership)) + owners = stems_reconstruction.stems_data.assignments_by_channel[channel_name] + + runs = envelopes.ownership[channel_name].runs + + assert [run.stem_id for run in runs] == [ + owner for index, owner in enumerate(owners) if index == 0 or owner != owners[index - 1] + ] + + def test_the_stretches_stay_within_the_bars_they_stand_under( + self, + stems_reconstruction: Reconstruction, + everything_heard_of: StemSelection, + ) -> None: + """A stretch names a frame the document records, and the bars may draw one more. + + An export releases the note it ends on, so a dimension's last item can stand past the + frames the record holds; the stretches stop where the record does. + """ + envelopes = heard_envelopes(stems_reconstruction, everything_heard_of) + + for channel_name, lane in envelopes.ownership.items(): + recorded = len(stems_reconstruction.stems_data.assignments_by_channel[channel_name]) + assert lane.runs[-1].end_frame == min(recorded, envelopes[channel_name].frame_count) + assert lane.runs[0].start_frame == 0 diff --git a/tests/unit/sampletones_application/logic/reconstruction/test_feature.py b/tests/unit/sampletones_application/logic/reconstruction/test_feature.py deleted file mode 100644 index 3a3b50a6d..000000000 --- a/tests/unit/sampletones_application/logic/reconstruction/test_feature.py +++ /dev/null @@ -1,93 +0,0 @@ -from __future__ import annotations - -from typing import Callable - -import pytest - -from sampletones_application.logic.reconstruction.feature import FeatureData -from sampletones_core.constants.enums import ChannelName -from sampletones_core.exporters import Features -from sampletones_core.reconstructions import Reconstruction -from sampletones_core.reconstructions.reconstruction.stems.selection import StemSelection - - -@pytest.fixture -def reconstruction( - reconstruction_factory: Callable[[], Reconstruction], -) -> Reconstruction: - return reconstruction_factory() - - -@pytest.fixture -def everything_heard(reconstruction: Reconstruction) -> StemSelection: - """The reader listening to every recording on every channel, as a fresh document reads.""" - return StemSelection.everywhere( - frozenset(reconstruction.stems_data.config.entries_by_id), - ChannelName.items(), - ) - - -@pytest.fixture -def feature_data(reconstruction: Reconstruction, everything_heard: StemSelection) -> FeatureData: - return FeatureData.heard(reconstruction, everything_heard) - - -class TestTheEnvelopesOfWhatIsHeard: - def test_every_generator_answers_with_an_entry( - self, - feature_data: FeatureData, - ) -> None: - assert set(feature_data.channels.keys()) == set(ChannelName.items()) - - def test_a_channel_standing_by_carries_empty_envelopes( - self, - reconstruction: Reconstruction, - feature_data: FeatureData, - ) -> None: - """A channel the reconstruction leaves silent is loaded describing no frame.""" - standing_by = set(ChannelName.items()) - set(reconstruction.playing_channels) - assert standing_by - assert all(not feature_data[channel_name].has_frames for channel_name in standing_by) - - def test_the_envelopes_state_the_pitch_they_are_measured_against( - self, - feature_data: FeatureData, - ) -> None: - for features in feature_data.channels.values(): - assert features.initial_pitch is not None - - def test_hearing_every_recording_reads_the_document_itself( - self, - reconstruction: Reconstruction, - feature_data: FeatureData, - ) -> None: - """Each channel that sounds is read as the document writes it. - - A channel whose every frame rests reads as standing by instead, since the reading ends - where a channel last sounds. - """ - whole = reconstruction.export() - sounding = {name: features for name, features in whole.items() if features.has_frames} - assert sounding - - assert {name: feature_data[name] for name in sounding} == sounding - - def test_a_channel_no_recording_is_heard_on_describes_no_frame( - self, - reconstruction: Reconstruction, - ) -> None: - playing = next(iter(reconstruction.playing_channels)) - - feature_data = FeatureData.heard(reconstruction, StemSelection(channels={})) - - assert not feature_data[playing].has_frames - - -class TestFeatureDataQueries: - @pytest.mark.parametrize("channel_name", ChannelName.items(), ids=lambda name: name.value) - def test_every_channel_answers_with_its_features( - self, - feature_data: FeatureData, - channel_name: ChannelName, - ) -> None: - assert isinstance(feature_data[channel_name], Features) diff --git a/tests/unit/sampletones_application/logic/reconstruction/test_instruments.py b/tests/unit/sampletones_application/logic/reconstruction/test_instruments.py index 2dadabf22..e02a126ad 100644 --- a/tests/unit/sampletones_application/logic/reconstruction/test_instruments.py +++ b/tests/unit/sampletones_application/logic/reconstruction/test_instruments.py @@ -9,11 +9,14 @@ from sampletones_application.logic.project.controller import ProjectController from sampletones_application.logic.project.manager import ProjectManager from sampletones_application.logic.reconstruction.editor import InstrumentEditor -from sampletones_application.logic.reconstruction.feature import FeatureData +from sampletones_application.logic.reconstruction.envelopes import heard_envelopes from sampletones_application.logic.reconstruction.instruments import ( ReconstructionInstrumentsLogic, ) from sampletones_application.logic.reconstruction.manager import ReconstructionManager +from sampletones_application.view_model.reconstruction.envelopes import ( + ChannelEnvelopesViewModel, +) from sampletones_application.view_model.reconstruction.instruments import ( ReconstructionInstrumentsViewModel, ) @@ -29,12 +32,13 @@ from tests.suite.stems import everything_heard -def _heard_features(reconstruction: Reconstruction) -> FeatureData: +def _heard_features(reconstruction: Reconstruction) -> ChannelEnvelopesViewModel: """The envelopes of the whole document, which is what a fresh reader hears.""" - return FeatureData.heard(reconstruction, everything_heard(reconstruction)) + return heard_envelopes(reconstruction, everything_heard(reconstruction)) HISTORY_BUDGET: Final[int] = 16 +NEW_PITCH: Final[int] = 61 def _editor( @@ -116,10 +120,10 @@ def test_with_features_fires_on_feature_data_changed_with_data( ) -> None: feature_data = _heard_features(reconstruction_factory()) mock_reconstruction_manager.current_features = feature_data - received: List[Optional[Dict[ChannelName, Features]]] = [] + received: List[Optional[ChannelEnvelopesViewModel]] = [] instruments_logic.on_feature_data_changed = received.append instruments_logic.update_display() - assert received == [feature_data.channels] + assert received == [ChannelEnvelopesViewModel(channels=feature_data.channels, ownership=feature_data.ownership)] def test_with_features_exposes_the_playing_generators( self, @@ -252,26 +256,30 @@ def test_schedules_reconstruction_update( self, instruments_logic: ReconstructionInstrumentsLogic, mock_reconstruction_manager: MagicMock, + reconstruction_factory: Callable[[], Reconstruction], ) -> None: + mock_reconstruction_manager.current_features = _heard_features(reconstruction_factory()) callback = MagicMock() instruments_logic.on_reconstruction_instrument_updated = callback - instruments_logic.handle_pitch_value_changed(ChannelName.PULSE1, 61) + instruments_logic.handle_pitch_value_changed(ChannelName.PULSE1, NEW_PITCH) callback.assert_called_once() def test_forwards_generator_pitch_feature_and_value( self, instruments_logic: ReconstructionInstrumentsLogic, mock_reconstruction_manager: MagicMock, + reconstruction_factory: Callable[[], Reconstruction], ) -> None: + mock_reconstruction_manager.current_features = _heard_features(reconstruction_factory()) callback = MagicMock() instruments_logic.on_reconstruction_instrument_updated = callback - instruments_logic.handle_pitch_value_changed(ChannelName.PULSE1, 61) - channel_name, feature_key, _ = callback.call_args.args - channel_features = mock_reconstruction_manager.current_features.channels[ChannelName.PULSE1] + instruments_logic.handle_pitch_value_changed(ChannelName.PULSE1, NEW_PITCH) + + channel_name, feature_key, features = callback.call_args.args assert channel_name == ChannelName.PULSE1 assert feature_key == FeatureKey.INITIAL_PITCH - channel_features.model_copy.assert_called_once_with(update={"initial_pitch": 61}) + assert features.initial_pitch == NEW_PITCH class TestReconstructionInstrumentsLogicHandleEnvelope: @@ -279,7 +287,9 @@ def test_an_edited_envelope_schedules_an_update( self, instruments_logic: ReconstructionInstrumentsLogic, mock_reconstruction_manager: MagicMock, + reconstruction_factory: Callable[[], Reconstruction], ) -> None: + mock_reconstruction_manager.current_features = _heard_features(reconstruction_factory()) callback = MagicMock() instruments_logic.on_reconstruction_instrument_updated = callback instruments_logic.handle_envelope_changed( @@ -389,13 +399,15 @@ def test_the_envelopes_are_drawn_under_the_channel_that_reads_them_all( self, instrument_logic: ReconstructionInstrumentsLogic, ) -> None: - received: List[Optional[Dict[ChannelName, Features]]] = [] + received: List[Optional[ChannelEnvelopesViewModel]] = [] instrument_logic.on_feature_data_changed = received.append instrument_logic.update_display() - assert received[-1] is not None - assert list(received[-1]) == [INSTRUMENT_CHANNEL] + envelopes = received[-1] + assert envelopes is not None + assert list(envelopes.channels) == [INSTRUMENT_CHANNEL] + assert envelopes.ownership == {} def test_an_envelope_edit_reaches_the_instrument_without_a_regeneration( self, diff --git a/tests/unit/sampletones_application/logic/reconstruction/test_ownership.py b/tests/unit/sampletones_application/logic/reconstruction/test_ownership.py new file mode 100644 index 000000000..67e02bbcb --- /dev/null +++ b/tests/unit/sampletones_application/logic/reconstruction/test_ownership.py @@ -0,0 +1,58 @@ +from typing import Dict, Final, List, Sequence, Tuple + +from sampletones_application.logic.reconstruction.ownership import ownership_lanes +from sampletones_core.constants.algorithm import RESTING_STEM_ID +from sampletones_core.constants.enums import ChannelName, bending_channels +from sampletones_core.reconstructions.reconstruction.stems.channel_assignment import ChannelAssignment +from sampletones_core.reconstructions.reconstruction.stems.data import StemsData +from sampletones_core.reconstructions.reconstructor.stems.configs.config import StemsConfig +from sampletones_core.reconstructions.reconstructor.stems.configs.entry import StemEntry +from sampletones_core.reconstructions.reconstructor.stems.configs.hierarchy import StemsHierarchy +from sampletones_core.reconstructions.reconstructor.stems.configs.settings import StemSettings + +CHANNEL: Final[ChannelName] = ChannelName.PULSE1 +STEM_A: Final[int] = 0 +STEM_B: Final[int] = 1 +STEM_CHANNELS: Final[List[ChannelName]] = [ChannelName.PULSE1, ChannelName.TRIANGLE] + + +def _stems_data(stem_ids: Sequence[int]) -> StemsData: + """A record of two recordings, the first channel holding ``stem_ids`` frame by frame.""" + entries = [ + StemEntry( + id=stem_id, + settings=StemSettings(channels=STEM_CHANNELS, bends=bending_channels(STEM_CHANNELS)), + ) + for stem_id in (STEM_A, STEM_B) + ] + return StemsData( + config=StemsConfig(entries=entries, hierarchy=StemsHierarchy(levels=[[STEM_A, STEM_B]])), + assignments=[ChannelAssignment(channel_name=CHANNEL, stem_ids=list(stem_ids))], + ) + + +def _lane_runs(stem_ids: Sequence[int]) -> Tuple[Tuple[int, int, int], ...]: + """The stretches the first channel divides into, each as its start, end and owner.""" + assignments: Dict[ChannelName, List[int]] = {CHANNEL: list(stem_ids)} + lanes = ownership_lanes(_stems_data(stem_ids), assignments, lambda _channel: {STEM_A, STEM_B}) + return tuple((run.start_frame, run.end_frame, run.stem_id) for run in lanes[CHANNEL].runs) + + +class TestWhatALaneDividesInto: + """A lane names an owner wherever the record does, and a rest answers to none. + + A resting stretch shows the ground of whatever surface carries the lane, so a stretch painted + for it would take the ribbon's own ground onto a plot answering to a different color. + """ + + def test_each_recording_takes_the_stretch_it_holds(self) -> None: + assert _lane_runs([STEM_A, STEM_A, STEM_B]) == ((0, 2, STEM_A), (2, 3, STEM_B)) + + def test_a_resting_stretch_takes_none_of_its_own(self) -> None: + assert _lane_runs([STEM_A, RESTING_STEM_ID, STEM_B]) == ((0, 1, STEM_A), (2, 3, STEM_B)) + + def test_a_channel_resting_throughout_carries_no_stretch(self) -> None: + assert _lane_runs([RESTING_STEM_ID, RESTING_STEM_ID]) == () + + def test_a_trailing_rest_leaves_the_recordings_where_they_stand(self) -> None: + assert _lane_runs([STEM_A, STEM_A, RESTING_STEM_ID]) == ((0, 2, STEM_A),) diff --git a/tests/unit/sampletones_application/logic/reconstruction/test_reconstruction.py b/tests/unit/sampletones_application/logic/reconstruction/test_reconstruction.py index 46eb7a275..a5f109285 100644 --- a/tests/unit/sampletones_application/logic/reconstruction/test_reconstruction.py +++ b/tests/unit/sampletones_application/logic/reconstruction/test_reconstruction.py @@ -8,12 +8,15 @@ from sampletones_application.exports import ExportBackends from sampletones_application.logic.reconstruction.data import ReconstructionData -from sampletones_application.logic.reconstruction.feature import FeatureData +from sampletones_application.logic.reconstruction.envelopes import heard_envelopes from sampletones_application.logic.reconstruction.listening import StemListening from sampletones_application.logic.reconstruction.manager import ReconstructionManager from sampletones_application.logic.reconstruction.reconstruction import ( ReconstructionPanelLogic, ) +from sampletones_application.view_model.reconstruction.envelopes import ( + ChannelEnvelopesViewModel, +) from sampletones_application.view_model.reconstruction.paths.state import ( ReconstructionPathState, ) @@ -23,7 +26,7 @@ from sampletones_application.view_model.shared.ownership import OwnershipRibbonViewModel from sampletones_core.audio import write_wave from sampletones_core.configs import Config -from sampletones_core.constants.algorithm import AUTHORED_STEM_ID, RESTING_STEM_ID +from sampletones_core.constants.algorithm import AUTHORED_STEM_ID from sampletones_core.constants.enums import AudioSourceType, ChannelName, bending_channels from sampletones_core.exports.format import ExportFormat from sampletones_core.instructions import PulseInstruction, TriangleInstruction @@ -99,7 +102,7 @@ def _open(manager: MagicMock, reconstruction_data: ReconstructionData) -> None: def _refresh_features(manager: MagicMock) -> None: """Reads the envelopes of what is heard, the way the manager's own refresh does.""" if manager.current_reconstruction is not None: - manager.current_features = FeatureData.heard( + manager.current_features = heard_envelopes( manager.current_reconstruction.reconstruction, manager.listening.selection, ) @@ -1344,12 +1347,27 @@ def test_the_lanes_stand_where_nothing_is_switched_on( assert all(lane.runs == () for lane in ribbon.lanes) assert ribbon.is_drawn - def test_a_recording_switched_off_leaves_its_stretches_resting( + def test_a_stretch_the_reader_hears_stands_under_its_recording( + self, + panel_logic: ReconstructionPanelLogic, + mock_reconstruction_manager: MagicMock, + stems_data: ReconstructionData, + ) -> None: + _open(mock_reconstruction_manager, stems_data) + received = self._ribbons(panel_logic) + + panel_logic.display_reconstruction() + + pulse_lane = next(lane for lane in received[-1].lanes if lane.channel_name == ChannelName.PULSE1) + assert [(run.stem_id, run.heard) for run in pulse_lane.runs] == [(0, True)] + + def test_a_recording_switched_off_keeps_its_stretch_under_its_own_name( self, panel_logic: ReconstructionPanelLogic, mock_reconstruction_manager: MagicMock, stems_data: ReconstructionData, ) -> None: + """A stretch names the recording holding it whether or not the reader is listening to it.""" _open(mock_reconstruction_manager, stems_data) panel_logic.display_reconstruction() received = self._ribbons(panel_logic) @@ -1357,7 +1375,7 @@ def test_a_recording_switched_off_leaves_its_stretches_resting( panel_logic.set_stem_channels(0, frozenset()) pulse_lane = next(lane for lane in received[-1].lanes if lane.channel_name == ChannelName.PULSE1) - assert [run.stem_id for run in pulse_lane.runs] == [RESTING_STEM_ID] + assert [(run.stem_id, run.heard) for run in pulse_lane.runs] == [(0, False)] def test_a_document_answering_to_one_recording_offers_no_lanes( self, diff --git a/tests/unit/sampletones_application/ui/elements/graphs/test_bar.py b/tests/unit/sampletones_application/ui/elements/graphs/test_bar.py new file mode 100644 index 000000000..0c4dbde3d --- /dev/null +++ b/tests/unit/sampletones_application/ui/elements/graphs/test_bar.py @@ -0,0 +1,132 @@ +from typing import Final, List, Optional, Tuple +from unittest.mock import patch + +import numpy as np +import pytest + +from sampletones_application.ui.elements.graphs import bar as bar_module +from sampletones_application.ui.elements.graphs.bar import GUIBarGraph +from sampletones_application.ui.elements.graphs.layers.bar import BarLayer +from sampletones_application.utils.palette.colors.literal import LiteralColor + +BAND_SHARE: Final[float] = 0.2 +BAR_VALUES: Final[Tuple[int, ...]] = (5, 9, 12, 3) +BAR_WEIGHT: Final[float] = 0.8 +DATA_RANGE: Final[Tuple[int, int]] = (0, 15) +HOVER_TAG: Final[str] = "bar.hover" +LAYER_NAME: Final[str] = "volume" +PLOT_RANGE: Final[Tuple[float, float]] = (-2.0, 17.0) +PLOT_TAG: Final[str] = "bar.plot" +PRESSED_BAR: Final[int] = 1 +PRESSED_VALUE: Final[float] = 6.0 + + +def _graph() -> GUIBarGraph: + """A bar plot holding one dimension's values, built as the panel builds it.""" + graph = GUIBarGraph.__new__(GUIBarGraph) + graph.plot_tag = PLOT_TAG + graph.hover_bar_tag = HOVER_TAG + graph.data_range = DATA_RANGE + graph.y_axis_tag = "bar.y" + graph.x_axis_tag = "bar.x" + graph.x_range = (-0.5, float(len(BAR_VALUES))) + graph.y_range = PLOT_RANGE + graph._default_y_range = PLOT_RANGE + graph.current_data = np.array(BAR_VALUES) + graph.layers = { + LAYER_NAME: BarLayer( + data=np.array(BAR_VALUES), + name=LAYER_NAME, + color=LiteralColor(value=(255, 255, 255, 255)), + bar_weight=BAR_WEIGHT, + ) + } + graph._draw_stroke = None + graph.on_bar_point_clicked = None + graph.on_bar_point_hovered = None + return graph + + +def _reserve_band(graph: GUIBarGraph, share: float) -> Tuple[float, float]: + """The band the plot keeps beneath its bars, with the axes it locks left to DearPyGui.""" + with ( + patch.object(bar_module.dpg, "set_axis_limits"), + patch.object(bar_module.dpg, "set_axis_limits_constraints"), + ): + return graph.reserve_band(share) + + +def _press( + graph: GUIBarGraph, + monkeypatch: pytest.MonkeyPatch, + position: Tuple[float, float], +) -> None: + """One frame of the left button held with the pointer standing at ``position`` in the plot. + + The plot's own items are DearPyGui's, so the reading answers that none of them is built and + the case reads the values the press leaves behind. + """ + monkeypatch.setattr(bar_module.dpg, "does_item_exist", lambda tag: False) + monkeypatch.setattr(bar_module, "dpg_is_item_hovered", lambda tag: True) + monkeypatch.setattr(bar_module, "dpg_configure_item", lambda tag, **kwargs: None) + monkeypatch.setattr(bar_module.dpg, "is_key_down", lambda key: False) + monkeypatch.setattr(bar_module.dpg, "get_plot_mouse_pos", lambda: position) + monkeypatch.setattr(bar_module.dpg, "is_mouse_button_down", lambda button: True) + monkeypatch.setattr(bar_module.dpg, "is_mouse_button_clicked", lambda button: False) + graph._on_mouse_action(PLOT_TAG) + + +class TestPressingTheBars: + """A press on the grid the bars stand on writes the value it lands at.""" + + def test_a_press_writes_the_value_it_lands_at(self, monkeypatch: pytest.MonkeyPatch) -> None: + graph = _graph() + + _press(graph, monkeypatch, (PRESSED_BAR + 0.5, PRESSED_VALUE)) + + expected = list(BAR_VALUES) + expected[PRESSED_BAR] = int(PRESSED_VALUE) + assert list(graph.layers[LAYER_NAME].y_data) == expected + + +class TestPressingTheBandBeneathTheBars: + """A band reserved beneath the bars reads which stretch belongs to whom. + + The band stands inside the plot the bars are drawn in, so a press meant for what it shows + reaches the same handler; the values above it are the reader's and stay as they stand. + """ + + def test_a_press_inside_the_band_leaves_the_values_standing(self, monkeypatch: pytest.MonkeyPatch) -> None: + graph = _graph() + band_low, band_high = _reserve_band(graph, BAND_SHARE) + + _press(graph, monkeypatch, (PRESSED_BAR + 0.5, (band_low + band_high) / 2)) + + assert list(graph.layers[LAYER_NAME].y_data) == list(BAR_VALUES) + + def test_a_press_inside_the_band_still_reports_the_bar_it_stands_under( + self, + monkeypatch: pytest.MonkeyPatch, + ) -> None: + graph = _graph() + band_low, band_high = _reserve_band(graph, BAND_SHARE) + hovered: List[Tuple[Optional[str], Optional[int]]] = [] + graph.on_bar_point_hovered = lambda name, index: hovered.append((name, index)) + + _press(graph, monkeypatch, (PRESSED_BAR + 0.5, (band_low + band_high) / 2)) + + assert hovered == [(LAYER_NAME, PRESSED_BAR)] + + def test_a_press_above_the_band_writes_as_it_did_before_the_band( + self, + monkeypatch: pytest.MonkeyPatch, + ) -> None: + """Reserving a band lowers what the plot spans and leaves the bars every value they reach.""" + graph = _graph() + _reserve_band(graph, BAND_SHARE) + + _press(graph, monkeypatch, (PRESSED_BAR + 0.5, PRESSED_VALUE)) + + expected = list(BAR_VALUES) + expected[PRESSED_BAR] = int(PRESSED_VALUE) + assert list(graph.layers[LAYER_NAME].y_data) == expected diff --git a/tests/unit/sampletones_application/ui/elements/graphs/test_ribbon.py b/tests/unit/sampletones_application/ui/elements/graphs/test_ribbon.py index bb964886f..c7e7a4e7b 100644 --- a/tests/unit/sampletones_application/ui/elements/graphs/test_ribbon.py +++ b/tests/unit/sampletones_application/ui/elements/graphs/test_ribbon.py @@ -20,6 +20,7 @@ ) AUTHORED_COLOR: Final[ColorRGBA] = (180, 140, 240, 255) REST_COLOR: Final[ColorRGBA] = (40, 40, 48, 255) +LEFT_OUT_FRACTION: Final[float] = 0.4 FRAME_LENGTH: Final[int] = 4 @@ -29,19 +30,26 @@ def stem_colors() -> StemColors: recordings=tuple(LiteralColor(value) for value in RECORDING_COLORS), authored=LiteralColor(AUTHORED_COLOR), rest=LiteralColor(REST_COLOR), + left_out_fraction=LEFT_OUT_FRACTION, ) def _runs(*entries: Tuple[int, int, int, int]) -> Tuple[OwnershipRunViewModel, ...]: return tuple( - OwnershipRunViewModel(start_frame=start, end_frame=end, stem_id=stem_id, position=position) + OwnershipRunViewModel( + start_frame=start, + end_frame=end, + stem_id=stem_id, + position=position, + heard=True, + ) for start, end, stem_id, position in entries ) class TestTheColorARecordingIsKnownBy: def test_a_recording_takes_the_color_of_the_place_it_holds(self, stem_colors: StemColors) -> None: - assert stem_colors.for_stem(7, 1).rgba == LiteralColor(RECORDING_COLORS[1]).rgba + assert stem_colors.for_stem(7, 1, heard=True).rgba == LiteralColor(RECORDING_COLORS[1]).rgba def test_the_list_starts_over_for_a_document_holding_more_recordings( self, @@ -50,10 +58,34 @@ def test_the_list_starts_over_for_a_document_holding_more_recordings( assert stem_colors.for_position(len(RECORDING_COLORS)).rgba == stem_colors.for_position(0).rgba def test_the_frames_the_reader_wrote_take_a_color_of_their_own(self, stem_colors: StemColors) -> None: - assert stem_colors.for_stem(AUTHORED_STEM_ID, 0).rgba == LiteralColor(AUTHORED_COLOR).rgba + assert stem_colors.for_stem(AUTHORED_STEM_ID, 0, heard=True).rgba == LiteralColor(AUTHORED_COLOR).rgba def test_a_resting_frame_shows_the_ground(self, stem_colors: StemColors) -> None: - assert stem_colors.for_stem(RESTING_STEM_ID, 0).rgba == LiteralColor(REST_COLOR).rgba + assert stem_colors.for_stem(RESTING_STEM_ID, 0, heard=True).rgba == LiteralColor(REST_COLOR).rgba + + +class TestHowSolidlyAStretchPaints: + """A recording left out keeps its color and carries it faded, so a stretch always names an owner.""" + + def test_a_recording_left_out_keeps_its_color(self, stem_colors: StemColors) -> None: + left_out = stem_colors.for_stem(7, 1, heard=False).rgba + + assert left_out[:3] == LiteralColor(RECORDING_COLORS[1]).rgba[:3] + + def test_a_recording_left_out_carries_less_of_its_opacity(self, stem_colors: StemColors) -> None: + heard = stem_colors.for_stem(7, 1, heard=True).rgba + + assert stem_colors.for_stem(7, 1, heard=False).rgba[3] < heard[3] + + def test_the_frames_the_reader_wrote_fade_like_any_other(self, stem_colors: StemColors) -> None: + left_out = stem_colors.for_stem(AUTHORED_STEM_ID, 0, heard=False).rgba + + assert left_out[:3] == LiteralColor(AUTHORED_COLOR).rgba[:3] + assert left_out[3] < LiteralColor(AUTHORED_COLOR).rgba[3] + + def test_a_resting_stretch_shows_the_ground_whoever_is_listening(self, stem_colors: StemColors) -> None: + """A rest answers to no recording, so every reader hears it.""" + assert stem_colors.for_stem(RESTING_STEM_ID, 0, heard=False).rgba == LiteralColor(REST_COLOR).rgba class TestWhatTheRibbonStandsFor: diff --git a/tests/unit/sampletones_application/ui/elements/test_window.py b/tests/unit/sampletones_application/ui/elements/test_window.py index 2d3cf79cd..8f5e2c594 100644 --- a/tests/unit/sampletones_application/ui/elements/test_window.py +++ b/tests/unit/sampletones_application/ui/elements/test_window.py @@ -4,24 +4,30 @@ import dearpygui.dearpygui as dpg import pytest +from sampletones_application.layout.primitives import DEARPYGUI_MAXIMUM_WINDOW_SIZE, DialogGeometry from sampletones_application.ui.elements.window import GUIWindow from sampletones_shared.types.callback import VoidCallback MODULE: Final[str] = "sampletones_application.ui.elements.window" TAG: Final[str] = "test.dialog.window.probe" STATED_WIDTH: Final[int] = 460 -CONTENT_HEIGHT: Final[int] = 0 +STATED_HEIGHT: Final[int] = 200 +VIEWPORT_WIDTH: Final[int] = 1280 +VIEWPORT_HEIGHT: Final[int] = 800 class ProbeWindow(GUIWindow): """A dialog whose content stretches across the window, the shape a stated width has to hold.""" - def __init__(self, on_close: Optional[VoidCallback]) -> None: + def __init__( + self, + on_close: Optional[VoidCallback], + geometry: Optional[DialogGeometry] = None, + ) -> None: self._on_close = on_close super().__init__( tag=TAG, - width=STATED_WIDTH, - height=CONTENT_HEIGHT, + geometry=geometry if geometry is not None else DialogGeometry(width=STATED_WIDTH, height=STATED_HEIGHT), ) def prepare(self, *_args: Any, **_kwargs: Any) -> None: @@ -50,11 +56,79 @@ def test_the_window_holds_the_width_it_states(self, dpg_context: None) -> None: assert dpg.get_item_configuration(TAG)["width"] == STATED_WIDTH - def test_the_window_takes_no_size_from_its_content(self, dpg_context: None) -> None: - """A window measuring itself against stretched content loses a pixel of width every frame.""" + def test_the_window_holds_its_stated_width_both_ways(self, dpg_context: None) -> None: + """A window free to widen, holding content measured against its width, widens every frame. + + The probe's combo stretches across the window, so the width it asks for follows the width + the window has; holding the window to one width is what leaves the two agreeing. + """ ProbeWindow(on_close=None).create_window() - assert dpg.get_item_configuration(TAG)["autosize"] is False + configuration = dpg.get_item_configuration(TAG) + assert configuration["autosize"] is True + assert configuration["min_size"][0] == STATED_WIDTH + assert configuration["max_size"][0] == STATED_WIDTH + + def test_the_window_leaves_its_height_to_what_it_holds(self, dpg_context: None) -> None: + """A prompt wrapping over several lines grows down, so nothing caps the height.""" + ProbeWindow(on_close=None).create_window() + + assert dpg.get_item_configuration(TAG)["max_size"][1] == DEARPYGUI_MAXIMUM_WINDOW_SIZE + + def test_the_window_opens_at_the_height_it_states(self, dpg_context: None) -> None: + """A dialog holding more than it states grows, so the stated height is where it starts.""" + ProbeWindow(on_close=None).create_window() + + assert dpg.get_item_configuration(TAG)["min_size"][1] == STATED_HEIGHT + + def test_a_window_stating_no_height_leaves_its_content_to_settle_it(self, dpg_context: None) -> None: + ProbeWindow(on_close=None, geometry=DialogGeometry(width=STATED_WIDTH)).create_window() + + assert dpg.get_item_configuration(TAG)["min_size"][1] == 0 + + +class TestWhereADialogOpens: + """A dialog stating a height is placed before it is drawn, so it never appears off center.""" + + @staticmethod + def _shown(geometry: DialogGeometry) -> Any: + window = ProbeWindow(on_close=None, geometry=geometry) + with ( + patch(f"{MODULE}.ThemeRegistry"), + patch(f"{MODULE}.center_when_settled"), + patch.object(dpg, "get_viewport_client_width", return_value=VIEWPORT_WIDTH), + patch.object(dpg, "get_viewport_client_height", return_value=VIEWPORT_HEIGHT), + ): + window.show() + + return dpg.get_item_pos(TAG) + + def test_a_window_stating_a_height_stands_at_its_center(self, dpg_context: None) -> None: + position = self._shown(DialogGeometry(width=STATED_WIDTH, height=STATED_HEIGHT)) + + assert position == [ + (VIEWPORT_WIDTH - STATED_WIDTH) // 2, + (VIEWPORT_HEIGHT - STATED_HEIGHT) // 2, + ] + + def test_a_window_taller_than_the_viewport_keeps_its_title_bar_reachable(self, dpg_context: None) -> None: + position = self._shown(DialogGeometry(width=STATED_WIDTH, height=VIEWPORT_HEIGHT * 2)) + + assert position[1] == 0 + + def test_a_window_settling_its_own_height_is_left_to_the_correction(self, dpg_context: None) -> None: + """Nothing states where it goes until a frame has measured it, so the pass does the placing.""" + window = ProbeWindow(on_close=None, geometry=DialogGeometry(width=STATED_WIDTH)) + + with ( + patch(f"{MODULE}.ThemeRegistry"), + patch(f"{MODULE}.center_when_settled") as center_when_settled, + patch.object(dpg, "set_item_pos") as set_item_pos, + ): + window.show() + + set_item_pos.assert_not_called() + center_when_settled.assert_called_once_with(TAG) class TestCloseAffordance: @@ -122,6 +196,15 @@ class TestRaisingAWindow: """A window is raised from wherever a result reaches the screen, the callback drain between frames included, so opening one waits on no frame.""" + @pytest.fixture(name="viewport", autouse=True) + def viewport_fixture(self) -> Iterator[None]: + """Stands in for the viewport a window is centered against, which a suite draws none of.""" + with ( + patch.object(dpg, "get_viewport_client_width", return_value=VIEWPORT_WIDTH), + patch.object(dpg, "get_viewport_client_height", return_value=VIEWPORT_HEIGHT), + ): + yield + def test_opening_waits_on_no_frame(self, dpg_context: None) -> None: window = ProbeWindow(on_close=None) diff --git a/tests/unit/sampletones_application/ui/panels/dialogs/conftest.py b/tests/unit/sampletones_application/ui/panels/dialogs/conftest.py index 514f516d7..064cbda5c 100644 --- a/tests/unit/sampletones_application/ui/panels/dialogs/conftest.py +++ b/tests/unit/sampletones_application/ui/panels/dialogs/conftest.py @@ -1,4 +1,5 @@ from typing import Iterator +from unittest.mock import patch import dearpygui.dearpygui as dpg import pytest @@ -36,3 +37,14 @@ def dpg_context(layout_config: LayoutConfig) -> Iterator[None]: finally: ThemeRegistry.clear() dpg.destroy_context() + + +@pytest.fixture(autouse=True) +def viewport(layout_config: LayoutConfig) -> Iterator[None]: + """Stands in for the viewport a dialog is centered against, which a suite draws none of.""" + window = layout_config.general.window + with ( + patch.object(dpg, "get_viewport_client_width", return_value=window.width), + patch.object(dpg, "get_viewport_client_height", return_value=window.height), + ): + yield diff --git a/tests/unit/sampletones_application/ui/panels/dialogs/test_export.py b/tests/unit/sampletones_application/ui/panels/dialogs/test_export.py index 6789c261e..aa252a427 100644 --- a/tests/unit/sampletones_application/ui/panels/dialogs/test_export.py +++ b/tests/unit/sampletones_application/ui/panels/dialogs/test_export.py @@ -50,9 +50,8 @@ def render( progress: float = HALFWAY, traveling: bool = True, ) -> None: - """Builds the widget tree and draws the given state, the way an open window is kept up to date.""" - window.create_window() - window.update_view( + """Raises the window on the given state, the way the coordinator running the export does.""" + window.open( SongExportViewModel( phase=phase, stages=stages, diff --git a/tests/unit/sampletones_application/ui/panels/reconstruction/test_instruments_panel.py b/tests/unit/sampletones_application/ui/panels/reconstruction/test_instruments_panel.py index cc6b18501..ca03df15a 100644 --- a/tests/unit/sampletones_application/ui/panels/reconstruction/test_instruments_panel.py +++ b/tests/unit/sampletones_application/ui/panels/reconstruction/test_instruments_panel.py @@ -16,14 +16,12 @@ PALETTES_DIRECTORY, THEME_DIRECTORY, ) -from sampletones_application.tags.compose import compose_tag from sampletones_application.tags.general import ( TAG_GLOBAL_THEME_DEFAULT, TAG_GLOBAL_THEME_INPUT_WARNING, TAG_GLOBAL_THEME_INSTRUMENT_TABS, TAG_GLOBAL_THEME_INSTRUMENT_TABS_MUTED, ) -from sampletones_application.tags.graphs import SUF_GRAPH_RAW_DATA from sampletones_application.ui.elements.button import GUIButton from sampletones_application.ui.elements.panel import GUIPanel from sampletones_application.ui.elements.pitch_stepper import PitchStepperStyle @@ -42,8 +40,13 @@ ReconstructionInstrumentsViewModel, ) from sampletones_application.view_model.shared.footprint import VoiceFootprintViewModel +from sampletones_application.view_model.shared.ownership import ( + OwnershipLaneViewModel, + OwnershipRunViewModel, +) from sampletones_core.constants.enums import ChannelName, FeatureKey, GeneratorName from sampletones_core.constants.general import PITCH_BEND_MAX, PITCH_BEND_MIN +from sampletones_core.exporters.feature import Features from sampletones_core.features import CHANNEL_GENERATOR_KIND, supported_features from sampletones_core.features.envelope import Envelope from sampletones_core.formats.famitracker.footprint import InstrumentFootprint @@ -60,6 +63,18 @@ LARGEST_TRIANGLE: Final[InstrumentFootprint] = InstrumentFootprint(instrument_bytes=7, sequence_bytes=512) SILENT_INSTRUMENT: Final[InstrumentFootprint] = InstrumentFootprint(instrument_bytes=3, sequence_bytes=0) +PLOTTED_AXIS: Final[str] = "instruments.plot.y" +PLOTTED_BAND: Final[tuple] = (-5.0, -2.0) +PLOTTED_CHANNEL: Final[ChannelName] = ChannelName.PULSE1 +PLOTTED_PITCH: Final[int] = 60 +PLOTTED_LANE: Final[OwnershipLaneViewModel] = OwnershipLaneViewModel( + channel_name=PLOTTED_CHANNEL, + runs=( + OwnershipRunViewModel(start_frame=0, end_frame=4, stem_id=0, position=0, heard=True), + OwnershipRunViewModel(start_frame=4, end_frame=10, stem_id=1, position=1, heard=True), + ), +) + NOT_LOADED: Final[ReconstructionInstrumentsViewModel] = ReconstructionInstrumentsViewModel( reconstruction_loaded=False, playing_channels=frozenset(), @@ -157,6 +172,7 @@ def panel( pitch_stepper_style=PitchStepperStyle.from_general(layout_config.general), copy_width=layout_config.general.buttons.copy_width, feature_colors=layout_config.general.colors.features, + stem_colors=layout_config.general.colors.stems, layout_graphs=layout_config.graphs, language_manager=LanguageManager(LANG_EN), status_bar=MagicMock(), @@ -213,6 +229,86 @@ def test_each_dimension_carries_its_own_length( ] +class TestTheBandBeneathADimension: + """A dimension's band stands over the frames that dimension draws, and no further. + + A channel's readings trim to their own lengths, so a lane handed whole to each of them paints + a stretch over frames that reading never drew. + """ + + @staticmethod + def _features(volume_items: int, duty_items: int) -> Features: + """One instrument whose volume and duty cycle write different numbers of frames.""" + return Features( + initial_pitch=PLOTTED_PITCH, + volume=sequence(volume_items), + arpeggio=sequence(volume_items), + pitch=sequence(volume_items), + hi_pitch=sequence(volume_items), + duty_cycle=sequence(duty_items), + ) + + def _painted( + self, + panel: GUIReconstructionInstrumentsPanel, + monkeypatch: pytest.MonkeyPatch, + features: Features, + feature_key: FeatureKey, + ) -> Dict[str, object]: + """What the panel hands the ownership painter for one dimension of a two-recording lane.""" + plot = MagicMock() + plot.y_axis_tag = PLOTTED_AXIS + plot.reserve_band.return_value = PLOTTED_BAND + panel.channel_plots[PLOTTED_CHANNEL] = {feature_key: plot} + painted: Dict[str, object] = {} + + def paint(y_axis_tag: str, runs: object, **kwargs: object) -> None: + painted["runs"] = runs + + monkeypatch.setattr(panel._ownership, "paint", paint) + panel._update_generator_feature_display(PLOTTED_CHANNEL, features, feature_key, PLOTTED_LANE) + painted["share"] = plot.reserve_band.call_args.args[0] + return painted + + def test_a_dimension_reaching_the_whole_lane_carries_every_stretch( + self, + panel: GUIReconstructionInstrumentsPanel, + monkeypatch: pytest.MonkeyPatch, + ) -> None: + painted = self._painted(panel, monkeypatch, self._features(10, 10), FeatureKey.VOLUME) + + assert painted["runs"] == PLOTTED_LANE.runs + + def test_a_trimmed_dimension_ends_its_stretches_where_it_ends( + self, + panel: GUIReconstructionInstrumentsPanel, + monkeypatch: pytest.MonkeyPatch, + ) -> None: + painted = self._painted(panel, monkeypatch, self._features(10, 1), FeatureKey.DUTY_CYCLE) + + assert painted["runs"] == (PLOTTED_LANE.runs[0].model_copy(update={"end_frame": 1}),) + + def test_a_dimension_writing_nothing_gives_the_band_to_the_bars( + self, + panel: GUIReconstructionInstrumentsPanel, + monkeypatch: pytest.MonkeyPatch, + ) -> None: + painted = self._painted(panel, monkeypatch, self._features(10, 0), FeatureKey.DUTY_CYCLE) + + assert painted["runs"] == () + assert painted["share"] == 0.0 + + def test_a_dimension_the_lane_reaches_keeps_its_band( + self, + panel: GUIReconstructionInstrumentsPanel, + monkeypatch: pytest.MonkeyPatch, + layout_config: LayoutConfig, + ) -> None: + painted = self._painted(panel, monkeypatch, self._features(10, 10), FeatureKey.VOLUME) + + assert painted["share"] == layout_config.graphs.bar_plot.ownership_band + + class TestEditingASequence: """A bar redrawn on the plot restates the values; the item the dimension repeats from is its own.""" @@ -228,7 +324,6 @@ def test_a_redrawn_bar_states_the_values_it_leaves( ChannelName.PULSE1, FeatureKey.VOLUME, np.array([15, 4, 8], dtype=np.int8), - "plot", ) assert edited == [Envelope[int](items=(15, 4, 8))] @@ -249,7 +344,6 @@ def test_a_redrawn_bar_keeps_the_item_the_dimension_repeats_from( ChannelName.PULSE1, FeatureKey.VOLUME, np.array([15, 4, 8], dtype=np.int8), - "plot", ) assert edited == [Envelope[int](items=(15, 4, 8), loop_point=1)] @@ -269,10 +363,10 @@ def test_a_redrawn_bar_writes_the_dimension_out_with_its_point( ChannelName.PULSE1, FeatureKey.VOLUME, np.array([15, 4, 8], dtype=np.int8), - "plot", ) - assert written[compose_tag("plot", SUF_GRAPH_RAW_DATA)] == "15 | 4 8" + field_tag = panel._get_feature_text_tag(ChannelName.PULSE1, FeatureKey.VOLUME) + assert written[field_tag] == "15 | 4 8" class TestCopyingASequence: @@ -677,6 +771,7 @@ def test_another_tab_in_front_keeps_the_keys_from_the_panel( pitch_stepper_style=PitchStepperStyle.from_general(layout_config.general), copy_width=layout_config.general.buttons.copy_width, feature_colors=layout_config.general.colors.features, + stem_colors=layout_config.general.colors.stems, layout_graphs=layout_config.graphs, language_manager=LanguageManager(LANG_EN), status_bar=MagicMock(), @@ -696,6 +791,7 @@ def test_a_rail_put_away_keeps_the_keys_from_the_panel( pitch_stepper_style=PitchStepperStyle.from_general(layout_config.general), copy_width=layout_config.general.buttons.copy_width, feature_colors=layout_config.general.colors.features, + stem_colors=layout_config.general.colors.stems, layout_graphs=layout_config.graphs, initial_collapsed=True, language_manager=LanguageManager(LANG_EN), diff --git a/tests/unit/sampletones_application/ui/panels/reconstruction/test_plot.py b/tests/unit/sampletones_application/ui/panels/reconstruction/test_plot.py index 8b83884b4..07498d645 100644 --- a/tests/unit/sampletones_application/ui/panels/reconstruction/test_plot.py +++ b/tests/unit/sampletones_application/ui/panels/reconstruction/test_plot.py @@ -354,7 +354,7 @@ def _ribbon(*channels: ChannelName) -> OwnershipRibbonViewModel: lanes=tuple( OwnershipLaneViewModel( channel_name=channel_name, - runs=(OwnershipRunViewModel(start_frame=0, end_frame=2, stem_id=0, position=0),), + runs=(OwnershipRunViewModel(start_frame=0, end_frame=2, stem_id=0, position=0, heard=True),), ) for channel_name in channels ), diff --git a/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_confirmation.py b/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_confirmation.py index 4ad5efcc5..d468baf79 100644 --- a/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_confirmation.py +++ b/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_confirmation.py @@ -31,8 +31,7 @@ def window_fixture(dpg_context: None, layout_config: LayoutConfig) -> GUIConfirmationWindow: return GUIConfirmationWindow( tag=WINDOW_TAG, - width=layout_config.general.dialogs.default.width, - height=layout_config.general.dialogs.confirmation.height, + geometry=layout_config.general.dialogs.confirmation, wrap=layout_config.general.dialogs.default.width - 10, path_color=layout_config.general.colors.paths.default, path_hover_color=layout_config.general.colors.paths.hover, diff --git a/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_error.py b/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_error.py index d65bb9bfd..aa87f3b44 100644 --- a/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_error.py +++ b/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_error.py @@ -1,4 +1,4 @@ -from typing import Final +from typing import Final, List import dearpygui.dearpygui as dpg import pytest @@ -15,6 +15,7 @@ TAG_GLOBAL_DIALOG_ERROR, ) from sampletones_application.utils.gui.dialogs import get_dialog_tag +from sampletones_application.utils.gui.dialogs.windows import error as error_module from sampletones_application.utils.gui.dialogs.windows.error import ( GUIErrorDialogWindow, ) @@ -29,8 +30,8 @@ def window_fixture(dpg_context: None, layout_config: LayoutConfig) -> GUIErrorDialogWindow: return GUIErrorDialogWindow( tag=WINDOW_TAG, - width=layout_config.general.dialogs.error.width, - height=layout_config.general.dialogs.error.height, + geometry=layout_config.general.dialogs.error, + traceback_height=layout_config.general.dialogs.traceback_height, wrap=layout_config.general.dialogs.error.width - 10, language_manager=LANGUAGE_MANAGER, error_color=layout_config.general.colors.text.error, @@ -65,6 +66,20 @@ def test_the_traceback_toggle_flips_its_label(self, window: GUIErrorDialogWindow assert dpg.get_item_label(show_inner_tag) == LANGUAGE_MANAGER["global.traceback.label.hide"] + def test_the_traceback_toggle_centers_the_window_on_its_new_height( + self, + window: GUIErrorDialogWindow, + monkeypatch: pytest.MonkeyPatch, + ) -> None: + """Unfolding a traceback grows the report past the screen unless it is placed again.""" + centered: List[str] = [] + monkeypatch.setattr(error_module, "center_when_settled", centered.append) + render(window) + + press(compose_tag(WINDOW_TAG, SUF_BUTTON_SHOW_TRACEBACK)) + + assert centered == [WINDOW_TAG] + def test_ok_dismisses_the_prompt(self, window: GUIErrorDialogWindow) -> None: render(window) diff --git a/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_save_confirmation.py b/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_save_confirmation.py index d5d4b373f..c6ae7b588 100644 --- a/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_save_confirmation.py +++ b/tests/unit/sampletones_application/utils/gui/dialogs/windows/test_save_confirmation.py @@ -27,8 +27,7 @@ def window_fixture(dpg_context: None, layout_config: LayoutConfig) -> GUISaveConfirmationWindow: return GUISaveConfirmationWindow( tag=WINDOW_TAG, - width=layout_config.general.dialogs.default.width, - height=layout_config.general.dialogs.confirmation.height, + geometry=layout_config.general.dialogs.confirmation, wrap=layout_config.general.dialogs.default.width - 10, save_label="Save", cancel_label="Cancel", diff --git a/tests/unit/sampletones_application/utils/gui/test_align.py b/tests/unit/sampletones_application/utils/gui/test_align.py new file mode 100644 index 000000000..aad4eb3d5 --- /dev/null +++ b/tests/unit/sampletones_application/utils/gui/test_align.py @@ -0,0 +1,112 @@ +from typing import Final, List, Sequence, Tuple + +import pytest + +from sampletones_application.utils.gui import align as align_module +from sampletones_application.utils.gui.align import center_when_settled +from sampletones_application.utils.placement import centered_position +from sampletones_shared.types.callback import VoidCallback + +FORM_SIZES: Final[Tuple[Tuple[int, int], ...]] = ((420, 45), (420, 217), (420, 217)) +SETTLED_SIZE: Final[Tuple[int, int]] = (420, 217) +VIEWPORT: Final[Tuple[int, int]] = (1600, 1000) +WIDENING_STEP: Final[int] = 63 +WINDOW_TAG: Final[str] = "settings.window" + + +class Window: + """A window drawn at a stated size each frame, with the frames the pass asks for run by hand. + + A frame runs the callbacks armed for it and the readings taken during it answer with that + frame's size, which is what lets a case state a window's whole way to its final size. + """ + + def __init__( + self, + monkeypatch: pytest.MonkeyPatch, + sizes: Sequence[Tuple[int, int]], + *, + exists: bool = True, + ) -> None: + self.sizes = list(sizes) + self.frame = 0 + self.positions: List[List[int]] = [] + self._armed: List[VoidCallback] = [] + monkeypatch.setattr(align_module.FrameCallbackManager, "set_frame_callback", self._arm) + monkeypatch.setattr(align_module.dpg, "does_item_exist", lambda tag: exists) + monkeypatch.setattr(align_module.dpg, "get_item_rect_size", lambda tag: self.size) + monkeypatch.setattr(align_module.dpg, "set_item_pos", lambda tag, position: self.positions.append(position)) + monkeypatch.setattr(align_module.dpg, "get_viewport_client_width", lambda: VIEWPORT[0]) + monkeypatch.setattr(align_module.dpg, "get_viewport_client_height", lambda: VIEWPORT[1]) + + @property + def size(self) -> Tuple[int, int]: + return self.sizes[min(self.frame, len(self.sizes) - 1)] + + @property + def waiting(self) -> bool: + """Whether the pass has asked for another frame.""" + return bool(self._armed) + + def _arm(self, callback: VoidCallback, frame_count: int = 1) -> None: + self._armed.append(callback) + + def draw(self, frames: int) -> None: + """Draws frames while the pass still asks for one.""" + for _ in range(frames): + if not self._armed: + return + + armed, self._armed = self._armed, [] + for callback in armed: + callback() + + self.frame += 1 + + +def _centered(size: Tuple[int, int]) -> List[int]: + return list(centered_position((VIEWPORT[0] // 2, VIEWPORT[1] // 2), *size)) + + +class TestCenteringAWindowAsItTakesItsSize: + """A dialog stands centered from the frame it is first drawn in until it settles there.""" + + def test_a_form_is_centered_against_every_size_it_is_drawn_at(self, monkeypatch: pytest.MonkeyPatch) -> None: + """A form measures its fields on the second frame, and stands centered on both.""" + window = Window(monkeypatch, FORM_SIZES) + + center_when_settled(WINDOW_TAG) + window.draw(len(FORM_SIZES)) + + assert window.positions == [_centered(FORM_SIZES[0]), _centered(SETTLED_SIZE), _centered(SETTLED_SIZE)] + + def test_two_readings_that_agree_end_the_pass(self, monkeypatch: pytest.MonkeyPatch) -> None: + """A dialog can be dragged, so the pass lets go once the size it reads stops changing.""" + window = Window(monkeypatch, FORM_SIZES) + + center_when_settled(WINDOW_TAG) + window.draw(len(FORM_SIZES) + 5) + + assert not window.waiting + assert len(window.positions) == len(FORM_SIZES) + + def test_a_window_still_widening_is_read_again(self, monkeypatch: pytest.MonkeyPatch) -> None: + """The keybindings list widens over a dozen frames, and is centered on each of them.""" + widening = [(620 + WIDENING_STEP * step, 647) for step in range(13)] + [(620 + WIDENING_STEP * 12, 647)] + window = Window(monkeypatch, widening) + + center_when_settled(WINDOW_TAG) + window.draw(len(widening)) + + assert window.positions == [_centered(size) for size in widening] + assert not window.waiting + + def test_a_window_no_longer_drawn_ends_the_pass(self, monkeypatch: pytest.MonkeyPatch) -> None: + """A dialog closed before it settled leaves nothing to center.""" + window = Window(monkeypatch, FORM_SIZES, exists=False) + + center_when_settled(WINDOW_TAG) + window.draw(len(FORM_SIZES)) + + assert window.positions == [] + assert not window.waiting diff --git a/tests/unit/sampletones_application/view_model/shared/test_ownership.py b/tests/unit/sampletones_application/view_model/shared/test_ownership.py new file mode 100644 index 000000000..47d1dd9f7 --- /dev/null +++ b/tests/unit/sampletones_application/view_model/shared/test_ownership.py @@ -0,0 +1,62 @@ +from typing import Final, Tuple + +import pytest + +from sampletones_application.view_model.shared.ownership import ( + OwnershipLaneViewModel, + OwnershipRunViewModel, +) +from sampletones_core.constants.enums import ChannelName + +FIRST_STEM: Final[int] = 0 +SECOND_STEM: Final[int] = 1 + + +def _run(start_frame: int, end_frame: int, stem_id: int) -> OwnershipRunViewModel: + return OwnershipRunViewModel( + start_frame=start_frame, + end_frame=end_frame, + stem_id=stem_id, + position=stem_id, + heard=True, + ) + + +def _lane(*runs: OwnershipRunViewModel) -> OwnershipLaneViewModel: + return OwnershipLaneViewModel(channel_name=ChannelName.PULSE1, runs=runs) + + +class TestTheStretchesUnderAReading: + """A lane runs the length of its channel, and answers for the frames a reading draws. + + A channel's readings trim to their own lengths, so a lane handed whole to each of them would + paint a stretch over frames that reading never drew. + """ + + def test_a_reading_as_long_as_the_channel_takes_every_stretch(self) -> None: + lane = _lane(_run(0, 4, FIRST_STEM), _run(4, 10, SECOND_STEM)) + + assert lane.up_to(10) == lane.runs + + def test_a_stretch_crossing_the_reading_ends_where_the_reading_does(self) -> None: + lane = _lane(_run(0, 4, FIRST_STEM), _run(4, 10, SECOND_STEM)) + + assert lane.up_to(6) == (_run(0, 4, FIRST_STEM), _run(4, 6, SECOND_STEM)) + + def test_a_stretch_beyond_the_reading_stands_out_of_it(self) -> None: + lane = _lane(_run(0, 4, FIRST_STEM), _run(4, 10, SECOND_STEM)) + + assert lane.up_to(4) == (_run(0, 4, FIRST_STEM),) + + def test_a_reading_of_one_frame_keeps_the_stretch_over_it(self) -> None: + lane = _lane(_run(0, 4, FIRST_STEM), _run(4, 10, SECOND_STEM)) + + assert lane.up_to(1) == (_run(0, 1, FIRST_STEM),) + + def test_a_reading_writing_nothing_carries_no_stretch(self) -> None: + lane = _lane(_run(0, 4, FIRST_STEM), _run(4, 10, SECOND_STEM)) + + assert lane.up_to(0) == () + + def test_a_lane_with_no_stretches_answers_with_none(self) -> None: + assert _lane().up_to(10) == () diff --git a/tests/unit/sampletones_core/compatibility/reconstruction/test_v2_2.py b/tests/unit/sampletones_core/compatibility/reconstruction/test_v2_2.py index 3fb38fb8f..51f6b9651 100644 --- a/tests/unit/sampletones_core/compatibility/reconstruction/test_v2_2.py +++ b/tests/unit/sampletones_core/compatibility/reconstruction/test_v2_2.py @@ -160,9 +160,6 @@ def test_a_file_naming_no_recording_records_no_source(self) -> None: """A document detached from its origin names none, which a stored project carries.""" assert update({})[STEMS_DATA][SOURCES] == [] - def test_the_key_the_record_replaces_is_let_go_of(self) -> None: - assert AUDIO_FILEPATH not in update({AUDIO_FILEPATH: RECORDING})[STEMS_DATA] - class TestWhoHoldsEachFrame: """Rest and silence name the same frames, so the owners are read from the stream itself.""" diff --git a/tests/unit/sampletones_core/data/test_document.py b/tests/unit/sampletones_core/data/test_document.py new file mode 100644 index 000000000..334fc6b60 --- /dev/null +++ b/tests/unit/sampletones_core/data/test_document.py @@ -0,0 +1,84 @@ +from pathlib import Path +from typing import Final + +import msgpack +import pytest + +from sampletones_core.data.document import ( + DOCUMENT_MAGIC, + compress_document, + decompress_document, + open_document, +) + +RECORDS: Final[int] = 512 +FIELD: Final[str] = "leading" + + +def _payload() -> bytes: + """A payload of the shape a stored document holds: one map per record, keyed by name.""" + records = [{"on": index % 2 == 0, "pitch": 33, "volume": index % 16} for index in range(RECORDS)] + return bytes(msgpack.packb({FIELD: "2.2", "records": records}, use_bin_type=True)) + + +class TestWhatAStoredDocumentHolds: + def test_a_payload_comes_back_as_it_was_written(self) -> None: + payload = _payload() + + assert decompress_document(compress_document(payload)) == payload + + def test_a_payload_written_before_the_framing_reads_as_it_stands(self) -> None: + payload = _payload() + + assert decompress_document(payload) == payload + + def test_the_stored_bytes_carry_the_framing_magic(self) -> None: + assert compress_document(_payload()).startswith(DOCUMENT_MAGIC) + + def test_the_repeated_field_names_deflate_away(self) -> None: + payload = _payload() + + assert len(compress_document(payload)) < len(payload) + + def test_storing_one_payload_twice_writes_the_same_bytes(self) -> None: + """The framing states its timestamp, so a document's bytes follow from its payload alone.""" + payload = _payload() + + assert compress_document(payload) == compress_document(payload) + + def test_an_empty_payload_survives_the_round_trip(self) -> None: + assert decompress_document(compress_document(b"")) == b"" + + +class TestReadingADocumentAsAStream: + def test_a_stored_document_streams_as_its_payload(self, tmp_path: Path) -> None: + payload = _payload() + path = tmp_path / "document.bin" + path.write_bytes(compress_document(payload)) + + with open_document(path) as stream: + assert stream.read() == payload + + def test_a_document_written_before_the_framing_streams_as_it_stands(self, tmp_path: Path) -> None: + payload = _payload() + path = tmp_path / "document.bin" + path.write_bytes(payload) + + with open_document(path) as stream: + assert stream.read() == payload + + def test_the_stream_reaches_the_front_without_the_rest(self, tmp_path: Path) -> None: + payload = _payload() + path = tmp_path / "document.bin" + path.write_bytes(compress_document(payload)) + + with open_document(path) as stream: + unpacker = msgpack.Unpacker(stream, raw=False) + unpacker.read_map_header() + + assert unpacker.unpack() == FIELD + + def test_a_missing_document_is_reported(self, tmp_path: Path) -> None: + with pytest.raises(FileNotFoundError): + with open_document(tmp_path / "absent.bin"): + pass diff --git a/tests/unit/sampletones_core/data/test_stored.py b/tests/unit/sampletones_core/data/test_stored.py index d05c24715..e2b47c47f 100644 --- a/tests/unit/sampletones_core/data/test_stored.py +++ b/tests/unit/sampletones_core/data/test_stored.py @@ -3,6 +3,7 @@ import msgpack +from sampletones_core.data.document import compress_document from sampletones_core.data.stored import read_leading_fields LEADING: Final[str] = "leading" @@ -39,6 +40,29 @@ def test_the_read_ends_once_every_named_field_is_found(self, tmp_path: Path) -> assert read_leading_fields(path, frozenset({LEADING, FOLLOWING})) == {LEADING: 1, FOLLOWING: 2} +class TestAStoredDocumentsFraming: + def test_the_named_fields_read_through_the_framing(self, tmp_path: Path) -> None: + document = {LEADING: {"version": "2.1"}, FOLLOWING: [1, 2], TRAILING: b"\x00" * 4096} + payload = msgpack.packb(document, use_bin_type=True) + path = _stored(tmp_path / "document.bin", compress_document(payload)) + + fields = read_leading_fields(path, frozenset({LEADING, FOLLOWING})) + + assert fields == {LEADING: document[LEADING], FOLLOWING: document[FOLLOWING]} + + def test_a_document_whose_framing_is_cut_short_holds_no_fields(self, tmp_path: Path) -> None: + stored = compress_document(msgpack.packb({LEADING: "x" * 4096}, use_bin_type=True)) + path = _stored(tmp_path / "document.bin", stored[:24]) + + assert not read_leading_fields(path, frozenset({LEADING})) + + def test_a_document_whose_framing_is_damaged_holds_no_fields(self, tmp_path: Path) -> None: + stored = compress_document(msgpack.packb({LEADING: "x" * 4096}, use_bin_type=True)) + path = _stored(tmp_path / "document.bin", stored[:20] + b"\xff" * 256) + + assert not read_leading_fields(path, frozenset({LEADING})) + + class TestAFrontThatDecodesAsNoMap: def test_an_empty_file_holds_no_fields(self, tmp_path: Path) -> None: path = _stored(tmp_path / "document.bin", b"") diff --git a/tests/unit/sampletones_core/reconstructions/reconstruction/test_rendering.py b/tests/unit/sampletones_core/reconstructions/reconstruction/test_approximations.py similarity index 81% rename from tests/unit/sampletones_core/reconstructions/reconstruction/test_rendering.py rename to tests/unit/sampletones_core/reconstructions/reconstruction/test_approximations.py index c27564783..d98392728 100644 --- a/tests/unit/sampletones_core/reconstructions/reconstruction/test_rendering.py +++ b/tests/unit/sampletones_core/reconstructions/reconstruction/test_approximations.py @@ -4,12 +4,11 @@ import pytest from sampletones_core.configs import Config -from sampletones_core.constants.algorithm import AUTHORED_STEM_ID, RESTING_STEM_ID, UNIT_DRIVE +from sampletones_core.constants.algorithm import RESTING_STEM_ID, UNIT_DRIVE from sampletones_core.constants.enums import ChannelName, bending_channels -from sampletones_core.generators.render import render_instructions +from sampletones_core.generators.render import render_channels, render_instructions from sampletones_core.instructions import InstructionUnion, PulseInstruction from sampletones_core.reconstructions.reconstruction.reconstruction import Reconstruction -from sampletones_core.reconstructions.reconstruction.rendering import render_streams from sampletones_core.reconstructions.reconstruction.stems.channel_assignment import ChannelAssignment from sampletones_core.reconstructions.reconstruction.stems.data import StemsData from sampletones_core.reconstructions.reconstruction.stems.filter import filter_approximations @@ -101,47 +100,23 @@ def reconstruction() -> Reconstruction: class TestTheAudioAReconstructionAnswersWith: - """A channel sounds what its generator renders from its own stream, at the drive its owner gives it.""" + """A channel sounds what its generator renders from the stream it carries, drive or none.""" - def test_each_frame_stands_at_the_drive_its_owner_gives_the_channel( + def test_a_channel_sounds_exactly_what_its_instructions_render( self, reconstruction: Reconstruction, ) -> None: - config = reconstruction.config - bare = render_instructions(reconstruction.instructions[ChannelName.PULSE1], ChannelName.PULSE1, config) + """The document behind this stands at a drive off unit, and sounds its instructions all the same.""" + rendered = render_channels(reconstruction.instructions, reconstruction.config) - rendered = render_streams(reconstruction.instructions, reconstruction.stems_data, config) - - np.testing.assert_allclose(_frame(rendered[ChannelName.PULSE1], 0), _frame(bare, 0) * LOUD_DRIVE) - np.testing.assert_allclose(_frame(rendered[ChannelName.PULSE1], 1), _frame(bare, 1) * UNIT_DRIVE) + np.testing.assert_array_equal(reconstruction.approximations[ChannelName.PULSE1], rendered[ChannelName.PULSE1]) def test_a_resting_frame_sounds_nothing(self, reconstruction: Reconstruction) -> None: - rendered = render_streams(reconstruction.instructions, reconstruction.stems_data, reconstruction.config) - np.testing.assert_array_equal( - _frame(rendered[ChannelName.PULSE1], 2), + _frame(reconstruction.approximations[ChannelName.PULSE1], 2), np.zeros(reconstruction.config.library.frame_length, dtype=np.float32), ) - def test_a_frame_the_reader_wrote_stands_at_unit_drive(self) -> None: - authored = _reconstruction( - {ChannelName.PULSE1: [_pulse(60), _pulse(62)]}, - {ChannelName.PULSE1: [STEM_A, AUTHORED_STEM_ID]}, - ) - bare = render_instructions(authored.instructions[ChannelName.PULSE1], ChannelName.PULSE1, authored.config) - - rendered = render_streams(authored.instructions, authored.stems_data, authored.config) - - np.testing.assert_allclose(_frame(rendered[ChannelName.PULSE1], 1), _frame(bare, 1) * UNIT_DRIVE) - - def test_the_reconstruction_reads_the_same_audio_the_render_answers( - self, - reconstruction: Reconstruction, - ) -> None: - rendered = render_streams(reconstruction.instructions, reconstruction.stems_data, reconstruction.config) - - np.testing.assert_array_equal(reconstruction.approximations[ChannelName.PULSE1], rendered[ChannelName.PULSE1]) - def test_a_channel_standing_by_sounds_nothing_at_all(self) -> None: reconstruction = _reconstruction( {ChannelName.PULSE1: [_pulse(60)]}, diff --git a/tests/unit/sampletones_core/reconstructions/reconstruction/test_heard_instructions.py b/tests/unit/sampletones_core/reconstructions/reconstruction/test_heard_instructions.py index 792ba2e1d..0ead497e1 100644 --- a/tests/unit/sampletones_core/reconstructions/reconstruction/test_heard_instructions.py +++ b/tests/unit/sampletones_core/reconstructions/reconstruction/test_heard_instructions.py @@ -120,7 +120,11 @@ def test_the_whole_selection_reads_the_stream_itself(self) -> None: class TestWhereTheReadingEnds: - """The reading runs to its last sounding frame, so what is heard states what it costs.""" + """The reading runs to the last frame the reader hears, so what is drawn is what is written. + + A rest belongs to no recording and is always read, so the reading lets go of the stretch a + left-out recording held and keeps every frame the channel describes. + """ def test_a_trailing_frame_left_out_leaves_the_reading(self) -> None: reading = _reading( @@ -131,26 +135,33 @@ def test_a_trailing_frame_left_out_leaves_the_reading(self) -> None: assert reading == [_pulse(60)] - def test_a_trailing_rest_leaves_the_reading(self) -> None: + def test_a_trailing_rest_stands_in_the_reading(self) -> None: + """A rest is read whoever is listening, so the frames drawn are the frames written.""" reading = _reading( [_pulse(60), _silence()], [STEM_A, RESTING_STEM_ID], _heard(STEM_A), ) - assert reading == [_pulse(60)] + assert reading == [_pulse(60), _silence()] def test_a_channel_with_nothing_heard_reads_as_standing_by(self) -> None: reading = _reading([_pulse(60), _pulse(62)], [STEM_A, STEM_B], _heard()) assert reading == [] - def test_a_channel_that_only_rests_reads_as_standing_by(self) -> None: - """Every frame resting leaves nothing to read, whatever the reader is listening to.""" - reading = _reading([_silence(), _silence()], [RESTING_STEM_ID, RESTING_STEM_ID], _heard(STEM_A, STEM_B)) + def test_a_channel_whose_sound_is_left_out_lets_go_of_its_rests_too(self) -> None: + """A rest is no evidence the reader hears the channel, so the channel reads as standing by.""" + reading = _reading([_pulse(60), _silence()], [STEM_A, RESTING_STEM_ID], _heard()) assert reading == [] + def test_a_channel_that_only_rests_keeps_the_frames_it_describes(self) -> None: + """A channel written down to nothing stays in play, so every reader of it counts one channel.""" + reading = _reading([_silence(), _silence()], [RESTING_STEM_ID, RESTING_STEM_ID], _heard(STEM_A, STEM_B)) + + assert reading == [_silence(), _silence()] + def test_a_channel_standing_by_reads_as_it_stands(self) -> None: instructions: Dict[ChannelName, Sequence[InstructionUnion]] = {CHANNEL: []} diff --git a/tests/unit/sampletones_core/reconstructions/reconstruction/test_reconstruction.py b/tests/unit/sampletones_core/reconstructions/reconstruction/test_reconstruction.py index df602226d..62ee9c2ab 100644 --- a/tests/unit/sampletones_core/reconstructions/reconstruction/test_reconstruction.py +++ b/tests/unit/sampletones_core/reconstructions/reconstruction/test_reconstruction.py @@ -17,6 +17,7 @@ bending_channels, ) from sampletones_core.data import Metadata +from sampletones_core.data.document import DOCUMENT_MAGIC from sampletones_core.features import resting_held_features, resting_reference from sampletones_core.instructions import PulseInstruction from sampletones_core.reconstructions import Reconstruction @@ -264,6 +265,30 @@ def test_detached_source_round_trips_as_empty( assert loaded.audio_filepath == () + def test_the_stored_file_carries_the_framing( + self, + tmp_path: Path, + reconstruction_factory: ReconstructionFactory, + ) -> None: + reconstruction = reconstruction_factory() + path = tmp_path / "framed.stn" + + reconstruction.save(path) + + assert path.read_bytes().startswith(DOCUMENT_MAGIC) + assert path.stat().st_size < len(reconstruction.serialize()) + + def test_a_file_written_before_the_framing_still_loads( + self, + tmp_path: Path, + reconstruction_factory: ReconstructionFactory, + ) -> None: + reconstruction = reconstruction_factory() + path = tmp_path / "plain.stn" + path.write_bytes(reconstruction.serialize()) + + assert Reconstruction.load(path).id == reconstruction.id + class TestDetachSource: def test_detach_clears_the_source_location( @@ -293,6 +318,18 @@ def test_corrupt_binary_raises_invalid_values(self) -> None: source="corrupt.stn", ) + def test_a_damaged_framing_raises_a_load_error( + self, + tmp_path: Path, + reconstruction_factory: ReconstructionFactory, + ) -> None: + path = tmp_path / "damaged.stn" + reconstruction_factory().save(path) + path.write_bytes(path.read_bytes()[:32] + b"\xff" * 256) + + with pytest.raises(LoadReconstructionError): + Reconstruction.load(path) + class TestLoadFileAccess(BaseTestSuite): @dataclass(frozen=True, kw_only=True) diff --git a/tests/unit/sampletones_core/reconstructions/reconstructor/stems/conftest.py b/tests/unit/sampletones_core/reconstructions/reconstructor/stems/conftest.py index fc59d854f..31f8bcb26 100644 --- a/tests/unit/sampletones_core/reconstructions/reconstructor/stems/conftest.py +++ b/tests/unit/sampletones_core/reconstructions/reconstructor/stems/conftest.py @@ -1,4 +1,5 @@ -from typing import Dict, Final, List, Mapping, Sequence +from dataclasses import dataclass +from typing import Dict, Final, List, Mapping, Optional, Sequence import numpy as np import pytest @@ -52,6 +53,15 @@ def audible_instruction_of(library_data: InstructionLibraryData, generator: Gene return next(instruction for instruction in library_data.filter((generator.class_name(),)).keys() if instruction.on) +@dataclass(frozen=True) +class _Pick: + """A channel standing for the next pick: what it would play, and the cost it is ranked on.""" + + channel_name: ChannelName + head: ScoredCandidate + unit_cost: float + + def frame_objective_baseline( fragment: Fragment, channels: Dict[ChannelName, GeneratorUnion], @@ -62,26 +72,30 @@ def frame_objective_baseline( Every candidate of every free channel's kind is scored alone at ``drive`` by the frame's cost with the picks so far sounding beside it, the lowest free channel of a kind standing for the - kind. The pick lowering the frame's cost the most is taken while one exists; the channels no - pick lowers hold their silence; every channel is then scored once more with the others' heads + kind. The channel taken is the one reaching the lowest cost at unit drive while that cost + lowers the frame, and it sounds the head its column at ``drive`` carries; the channels no pick + reaches hold their silence; every channel is then scored once more with the others' heads sounding. A one-stem setup at full count driving every channel alike runs exactly this, which is what makes the two comparable frame by frame. """ heads: Dict[ChannelName, ScoredCandidate] = {} frame_cost = matcher.mix_cost(fragment, FrameMix.empty(fragment)) while True: - best = None + best: Optional[_Pick] = None for channel_name in _representatives(channels, heads): context = [head.contribution for head in heads.values()] - head = _column(fragment, channels[channel_name], context, matcher, drive)[0] - if head.instruction.on and head.cost < frame_cost and (best is None or head.cost < best[1].cost): - best = (channel_name, head) + generator = channels[channel_name] + column = _column(fragment, generator, context, matcher, drive) + unit_column = column if drive == UNIT_DRIVE else _column(fragment, generator, context, matcher, UNIT_DRIVE) + unit_cost = unit_column[0].cost + if column[0].instruction.on and unit_cost < frame_cost and (best is None or unit_cost < best.unit_cost): + best = _Pick(channel_name=channel_name, head=column[0], unit_cost=unit_cost) if best is None: break - heads[best[0]] = best[1] - frame_cost = best[1].cost + heads[best.channel_name] = best.head + frame_cost = best.unit_cost for channel_name in channels: if channel_name not in heads: @@ -131,7 +145,7 @@ def _scored_alone( ) -> ScoredCandidate: provider = matcher.candidate_provider mix = FrameMix.of(fragment, context) - power = provider.power_of(instruction) * drive**2 + power = provider.power_of(instruction) / drive**2 contribution = matcher.contribution(instruction, mix.residual_waveform(fragment), power, drive=drive) spectral = matcher.scorer.spectral_costs(fragment, provider.features_of((mix.power + power)[None, :])) cost = matcher.scorer.frame_costs( diff --git a/tests/unit/sampletones_core/reconstructions/reconstructor/stems/test_frame.py b/tests/unit/sampletones_core/reconstructions/reconstructor/stems/test_frame.py index 09233769e..61b2dde8d 100644 --- a/tests/unit/sampletones_core/reconstructions/reconstructor/stems/test_frame.py +++ b/tests/unit/sampletones_core/reconstructions/reconstructor/stems/test_frame.py @@ -7,6 +7,7 @@ from sampletones_core.configs import Config from sampletones_core.constants.algorithm import ( ALL_STEMS_CHANNEL_CAP, + MAX_DRIVE, SINGLE_STATE_LATTICE_WIDTH, UNIT_DRIVE, ) @@ -38,6 +39,7 @@ DEFAULT_CHANNELS: List[ChannelName] = [ChannelName.PULSE1, ChannelName.TRIANGLE, ChannelName.NOISE] LOUDER_STEM_SCALE: Final[float] = 4.0 LOUD_DRIVE: Final[float] = 2.0 +SWEPT_DRIVES: Final[Tuple[float, ...]] = (UNIT_DRIVE, LOUD_DRIVE, MAX_DRIVE) def _config( @@ -253,6 +255,75 @@ def test_a_level_of_its_own_takes_the_channel_first( assert [choice.stem_id for choice in assignment.choices] == [0] +class TestDrivesLeaveTheCompetitionStanding: + """A drive settles what a channel plays and leaves the stems competing where they stood. + + This is the contract ``docs/concepts/stems.md`` states of a drive: raising one lifts the part of the mix its own channel carries, while the stems beside it hold + the channels they held. + """ + + @staticmethod + def _stems_config(driven_stem_id: int, drive: float) -> StemsConfig: + """Two stems on one level reaching for the first pulse, one of them pushed to ``drive``.""" + held = [ChannelName.PULSE1] + return StemsConfig( + entries=[ + StemEntry( + id=stem_id, + settings=StemSettings( + channels=held, + bends=bending_channels(held), + drives={ChannelName.PULSE1: drive if stem_id == driven_stem_id else UNIT_DRIVE}, + channel_cap=1, + ), + ) + for stem_id in (0, 1) + ], + hierarchy=StemsHierarchy(levels=[[0, 1]], mode=HierarchyMode.STRICT), + ) + + def _winner( + self, + fragments: Dict[int, Fragment], + channels: Dict[ChannelName, GeneratorUnion], + matcher: FrameMatcher, + driven_stem_id: int, + drive: float, + ) -> int: + """The stem the first pulse reaches with ``driven_stem_id`` pushed to ``drive``.""" + assignment = assign_frame( + fragments, + self._stems_config(driven_stem_id, drive), + channels, + matcher, + SINGLE_STATE_LATTICE_WIDTH, + ) + return assignment.by_channel[ChannelName.PULSE1].stem_id + + @pytest.mark.parametrize("driven_stem_id", (0, 1)) + def test_the_channel_holds_its_stem_at_every_drive( + self, + driven_stem_id: int, + audible_fragments: List[Fragment], + channels: Dict[ChannelName, GeneratorUnion], + matcher: FrameMatcher, + ) -> None: + """Two recordings reaching for one channel keep it where it stood, whichever is pushed. + + The stems are ranked on the channel read at unit drive, so the winner follows the + recordings alone and a reader raising a drive hears that stem louder where it stood. The + sweep stands at and above the level the library is calibrated to, the range this + library's rows answer. + """ + assert len(audible_fragments) >= 2 + fragments = {0: audible_fragments[0], 1: audible_fragments[-1]} + standing = self._winner(fragments, channels, matcher, driven_stem_id, UNIT_DRIVE) + + winners = {drive: self._winner(fragments, channels, matcher, driven_stem_id, drive) for drive in SWEPT_DRIVES} + + assert winners == {drive: standing for drive in SWEPT_DRIVES} + + class TestColumns: """Every channel leaves the frame with the alternatives the decoder reads.""" diff --git a/tests/unit/sampletones_core/reconstructions/reconstructor/test_matching.py b/tests/unit/sampletones_core/reconstructions/reconstructor/test_matching.py index 8e54e60e8..0a5101330 100644 --- a/tests/unit/sampletones_core/reconstructions/reconstructor/test_matching.py +++ b/tests/unit/sampletones_core/reconstructions/reconstructor/test_matching.py @@ -7,7 +7,7 @@ import pytest from sampletones_core.configs import Config -from sampletones_core.constants.algorithm import UNIT_DRIVE +from sampletones_core.constants.algorithm import MAX_DRIVE, UNIT_DRIVE from sampletones_core.constants.enums import ChannelName, GeneratorClassName from sampletones_core.fft import Fragment, Window from sampletones_core.fft.features import FeatureExtractor @@ -29,6 +29,11 @@ AVERAGE_TOLERANCE: Final[float] = 0.05 SOUNDING_COST: Final[float] = 0.2 SILENT_COST: Final[float] = 0.5 +LADDER_PITCH: Final[int] = 60 +LADDER_DUTY_CYCLE: Final[int] = 0 +LADDER_STEP: Final[int] = 2 +QUIET_RUNG: Final[int] = 3 +MIDDLE_RUNG: Final[int] = 7 def _long_noise(library_data: InstructionLibraryData) -> NoiseInstruction: @@ -126,7 +131,7 @@ def test_a_phase_shifted_rendering_wins_its_class_at_near_zero_cost( assert column[0].instruction == audible_instruction assert column[0].cost == pytest.approx(0.0, abs=1e-3) - def test_a_driven_candidate_answers_a_target_playing_that_loud( + def test_a_driven_candidate_answers_a_target_it_sounds_louder_than( self, worker: ReconstructorWorker, library_data: InstructionLibraryData, @@ -135,9 +140,9 @@ def test_a_driven_candidate_answers_a_target_playing_that_loud( config: Config, window: Window, ) -> None: - """A candidate scored at a drive stands where its library sample played that loud stands.""" + """A candidate scored at a drive answers the target its library sample sounds that loud over.""" library_fragment = library_data[audible_instruction].get_fragment(0, config, window) - target = amplified(extractor, library_fragment, DRIVE) + target = amplified(extractor, library_fragment, UNIT_DRIVE / DRIVE) column = worker.matcher.score_column( target, @@ -255,7 +260,7 @@ def test_long_noise_costs_what_it_averages_to_over_its_shifts( to_numpy( criterion.temporal_loss( xp.asarray(synthetic_fragment.audio), - xp.asarray(library_fragment.get_fragment(shift, config, window).audio * DRIVE), + xp.asarray(library_fragment.get_fragment(shift, config, window).audio / DRIVE), ) )[0] ) @@ -266,7 +271,7 @@ def test_long_noise_costs_what_it_averages_to_over_its_shifts( contribution = worker.matcher.contribution( noise, np.asarray(synthetic_fragment.audio, dtype=np.float64), - worker.candidate_provider.power_of(noise) * DRIVE**2, + worker.candidate_provider.power_of(noise) / DRIVE**2, drive=DRIVE, ) @@ -296,6 +301,100 @@ def test_a_note_searched_for_costs_no_more_than_at_its_library_phase( ) +def _ladder_instructions(generator: GeneratorUnion) -> List[InstructionUnion]: + """One pulse pitch at every other volume it holds, with the channel's silence beside them. + + Stepping over the volumes keeps the whole ladder inside the spectral shortlist, so the + loudest rung stands before the full cost whatever drive the column is scored at. + """ + rungs = sorted( + ( + instruction + for instruction in generator.get_possible_instructions() + if instruction.on and instruction.pitch == LADDER_PITCH and instruction.duty_cycle == LADDER_DUTY_CYCLE + ), + key=lambda instruction: instruction.volume, + ) + return [*rungs[::LADDER_STEP], generator.get_instruction_type().null_instruction()] + + +def _ladder_target( + worker: ReconstructorWorker, + config: Config, + window: Window, + volume: int, +) -> Fragment: + """The frame the rung at ``volume`` sounds, which a drive is measured against.""" + rung = next( + instruction for instruction in worker.library_data.keys() if instruction.on and instruction.volume == volume + ) + return worker.library_data[rung].get_fragment(0, config, window) + + +def _winner(worker: ReconstructorWorker, target: Fragment, drive: float) -> InstructionUnion: + """The row a channel plays in ``target`` at ``drive``.""" + generator = worker.channels[ChannelName.PULSE1] + column = worker.matcher.score_column(target, generator, FrameMix.empty(target), drive=drive) + return column[0].instruction + + +def _loudest(worker: ReconstructorWorker) -> InstructionUnion: + """The loudest row the ladder holds, which is the level the channel saturates at.""" + return max( + (instruction for instruction in worker.library_data.keys() if instruction.on), + key=lambda instruction: instruction.volume, + ) + + +@pytest.fixture(scope="module") +def ladder_worker( + config: Config, + window: Window, + extractor: FeatureExtractor, + channels: Dict[ChannelName, GeneratorUnion], +) -> ReconstructorWorker: + """A run over one pulse pitch at a ladder of volumes, which is what a drive climbs.""" + generator = channels[ChannelName.PULSE1] + data: Dict[InstructionUnion, InstructionLibraryFragment[Any]] = { + instruction: InstructionLibraryFragment.create(generator, instruction, extractor) + for instruction in _ladder_instructions(generator) + } + return ReconstructorWorker( + config=config, + window=window, + channels={ChannelName.PULSE1: generator}, + library_data=InstructionLibraryData.create(config, data), + signal_length=WORKER_SIGNAL_LENGTH, + ) + + +class TestTheRowADriveReachesFor: + """A drive lifts what a channel reaches for, up to the loudest row the channel holds.""" + + def test_a_driven_channel_reaches_for_a_louder_row( + self, + ladder_worker: ReconstructorWorker, + config: Config, + window: Window, + ) -> None: + target = _ladder_target(ladder_worker, config, window, QUIET_RUNG) + + standing = _winner(ladder_worker, target, UNIT_DRIVE) + driven = _winner(ladder_worker, target, DRIVE) + + assert standing.volume < driven.volume < _loudest(ladder_worker).volume + + def test_a_drive_the_channel_cannot_reach_settles_on_its_loudest_row( + self, + ladder_worker: ReconstructorWorker, + config: Config, + window: Window, + ) -> None: + target = _ladder_target(ladder_worker, config, window, MIDDLE_RUNG) + + assert _winner(ladder_worker, target, MAX_DRIVE) == _loudest(ladder_worker) + + class TestColumnOf: def _column(self) -> tuple[ScoredCandidate, ...]: silence = Contribution.silence(1, 1) diff --git a/tests/unit/sampletones_core/reconstructions/reconstructor/test_phase.py b/tests/unit/sampletones_core/reconstructions/reconstructor/test_phase.py index 6be28a0dd..1d03c5e5d 100644 --- a/tests/unit/sampletones_core/reconstructions/reconstructor/test_phase.py +++ b/tests/unit/sampletones_core/reconstructions/reconstructor/test_phase.py @@ -48,7 +48,7 @@ def test_cross_correlation_reaches_the_sliding_rmse_optimum( class TestPhaseAlignerDrive: @pytest.mark.parametrize("aligner_class", [SlidingRmsePhaseAligner, CrossCorrelationPhaseAligner]) - def test_aligned_candidate_matches_a_drive_scaled_target( + def test_an_aligned_candidate_matches_the_target_it_sounds_louder_than( self, aligner_class: Type[PhaseAligner], config: Config, @@ -58,13 +58,13 @@ def test_aligned_candidate_matches_a_drive_scaled_target( ) -> None: """ The aligner searches and returns the candidate at the amplitude it competes - at, so a target that is a drive-scaled, phase-shifted rendering of the - candidate is reproduced exactly. + at, so a target the candidate sounds ``drive`` times as loud as, phase-shifted, + is reproduced exactly. """ aligner = aligner_class(config, window, library_data) library_fragment = library_data[audible_instruction] - target = DRIVE * library_fragment.get_fragment(library_fragment.length // 4, config, window).audio + target = library_fragment.get_fragment(library_fragment.length // 4, config, window).audio / DRIVE aligned = aligner.align(target, audible_instruction, DRIVE) assert _rmse(target, aligned) == pytest.approx(0.0, abs=1e-4) diff --git a/tests/unit/sampletones_core/reconstructions/test_progress.py b/tests/unit/sampletones_core/reconstructions/test_progress.py index 3e6194c80..8ff24c6b3 100644 --- a/tests/unit/sampletones_core/reconstructions/test_progress.py +++ b/tests/unit/sampletones_core/reconstructions/test_progress.py @@ -71,7 +71,7 @@ def label(self) -> str: TestCase(stage=ReconstructionStage.MATCHING, covered=0.0, expected=0.0), TestCase(stage=ReconstructionStage.MATCHING, covered=0.5, expected=0.5), TestCase(stage=ReconstructionStage.MATCHING, covered=WHOLE, expected=WHOLE), - TestCase(stage=ReconstructionStage.RENDERING, covered=WHOLE, expected=WHOLE), + TestCase(stage=ReconstructionStage.GATHERING, covered=WHOLE, expected=WHOLE), ) @pytest.mark.parametrize("test_case", test_cases, ids=lambda test_case: test_case.label) @@ -131,4 +131,4 @@ def test_a_caller_watching_nothing_hears_the_run_through(self) -> None: readings: Tuple[int, ...] = (0, FRAMES // 2, FRAMES) for completed in readings: - announce(silent_reporter, ReconstructionStage.RENDERING, completed, FRAMES) + announce(silent_reporter, ReconstructionStage.GATHERING, completed, FRAMES) diff --git a/tests/unit/sampletones_player/clock/test_schedule.py b/tests/unit/sampletones_player/clock/test_schedule.py index e60e8b1a7..bb859f203 100644 --- a/tests/unit/sampletones_player/clock/test_schedule.py +++ b/tests/unit/sampletones_player/clock/test_schedule.py @@ -14,12 +14,27 @@ MAX_STEP_WHOLE, NTSC_FRAME_RATE, ) -from sampletones_shared.constants.nes import MAX_NES_FREQUENCY, MIN_NES_FREQUENCY +from sampletones_shared.constants.nes import ( + MAX_NES_FREQUENCY, + MIN_NES_FREQUENCY, +) from tests.suite.base import BaseTestSuite from tests.suite.case import BaseAutolabelTestCase LONG_RUN_PLAY_CALLS: Final[int] = 36000 -NES_FREQUENCIES: Final[Tuple[int, ...]] = (15, 24, 25, 30, 50, 60, 100, 120, 200, 299, 300) +NES_FREQUENCIES: Final[Tuple[int, ...]] = ( + 15, + 24, + 25, + 30, + 50, + 60, + 100, + 120, + 200, + 299, + 300, +) def exact_rate(nes_frequency: int) -> Fraction: @@ -149,7 +164,10 @@ def test_the_step_fields_match(self, test_case: TestCase) -> None: @pytest.mark.parametrize("nes_frequency", NES_FREQUENCIES) def test_the_value_recomposes_the_fields(self, nes_frequency: int) -> None: step = PlaySchedule.from_parameters(nes_frequency).fixed_point_step - assert divmod(step.value, FIXED_POINT_SCALE) == (step.whole, step.fraction) + assert divmod(step.value, FIXED_POINT_SCALE) == ( + step.whole, + step.fraction, + ) @pytest.mark.parametrize("nes_frequency", NES_FREQUENCIES) def test_the_step_is_the_nearest_unit_to_the_exact_rate(self, nes_frequency: int) -> None: @@ -189,9 +207,15 @@ def test_the_accumulator_reaches_the_same_ticks(self, nes_frequency: int) -> Non def test_a_whole_step_needs_no_fraction(self) -> None: step = PlaySchedule(ticks_per_play_call=Fraction(3)).fixed_point_step - assert (step.whole, step.fraction, step.value) == (3, 0, 3 * FIXED_POINT_SCALE) - - def test_a_step_a_hair_under_a_whole_tick_carries_into_the_whole_byte(self) -> None: + assert (step.whole, step.fraction, step.value) == ( + 3, + 0, + 3 * FIXED_POINT_SCALE, + ) + + def test_a_step_a_hair_under_a_whole_tick_carries_into_the_whole_byte( + self, + ) -> None: """Rounding the fraction up spills into the whole byte, keeping both fields in range.""" rate = Fraction(2) - Fraction(1, 10 * FIXED_POINT_SCALE) step = PlaySchedule(ticks_per_play_call=rate).fixed_point_step @@ -235,7 +259,9 @@ def test_a_negative_call_count_is_rejected(self) -> None: with pytest.raises(ValueError, match="play_calls must be at least 0"): schedule.ticks_at(-1) - def test_a_negative_call_count_is_rejected_by_the_exact_schedule(self) -> None: + def test_a_negative_call_count_is_rejected_by_the_exact_schedule( + self, + ) -> None: schedule = PlaySchedule.from_parameters(60) with pytest.raises(ValueError, match="play_calls must be at least 0"): schedule.exact_ticks_at(-1) diff --git a/tests/unit/sampletones_player/clock/test_step.py b/tests/unit/sampletones_player/clock/test_step.py index 35572dbef..6c64ba496 100644 --- a/tests/unit/sampletones_player/clock/test_step.py +++ b/tests/unit/sampletones_player/clock/test_step.py @@ -32,7 +32,10 @@ def label(self) -> str: TestCase(fields=(0, MAX_STEP_FRACTION), expected=MAX_STEP_FRACTION), TestCase(fields=(1, 0), expected=FIXED_POINT_SCALE), TestCase(fields=(3, 21837), expected=3 * FIXED_POINT_SCALE + 21837), - TestCase(fields=(MAX_STEP_WHOLE, MAX_STEP_FRACTION), expected=FIXED_POINT_SCALE * (MAX_STEP_WHOLE + 1) - 1), + TestCase( + fields=(MAX_STEP_WHOLE, MAX_STEP_FRACTION), + expected=FIXED_POINT_SCALE * (MAX_STEP_WHOLE + 1) - 1, + ), ) @pytest.mark.parametrize("test_case", test_cases, ids=lambda test_case: test_case.label) @@ -44,7 +47,10 @@ def test_the_value_composes_the_fields(self, test_case: TestCase) -> None: def test_the_fields_read_back_off_the_value(self, test_case: TestCase) -> None: whole, fraction = test_case.fields step = FixedPointStep(whole=whole, fraction=fraction) - assert divmod(step.value, FIXED_POINT_SCALE) == (step.whole, step.fraction) + assert divmod(step.value, FIXED_POINT_SCALE) == ( + step.whole, + step.fraction, + ) @pytest.mark.parametrize("fraction", (-1, FIXED_POINT_SCALE)) def test_a_fraction_outside_the_word_is_rejected(self, fraction: int) -> None: diff --git a/tests/unit/sampletones_player/compression/dictionary/test_phrase.py b/tests/unit/sampletones_player/compression/dictionary/test_phrase.py index fc945a7ac..a18fc934b 100644 --- a/tests/unit/sampletones_player/compression/dictionary/test_phrase.py +++ b/tests/unit/sampletones_player/compression/dictionary/test_phrase.py @@ -4,6 +4,7 @@ from sampletones_player.compression.dictionary.phrase import Phrase from sampletones_player.specification.compression import ( MAX_PHRASE_LENGTH, + PHRASE_DEFAULT_SIZE, PHRASE_LENGTH_SIZE, PHRASE_TABLE_ENTRY_SIZE, ) @@ -16,14 +17,18 @@ class TestAPhraseIsAShapeRatherThanValues: def test_the_steps_read_the_body_pairwise(self) -> None: assert phrase(10, 12, 9).differences == bytes((2, 253)) - def test_a_shifted_phrase_keeps_the_steps_of_the_one_it_was_stored_from(self) -> None: + def test_a_shifted_phrase_keeps_the_steps_of_the_one_it_was_stored_from( + self, + ) -> None: assert phrase(70, 72, 69).differences == phrase(10, 12, 9).differences def test_a_phrase_covers_the_ticks_its_body_states(self) -> None: assert phrase(1, 2, 3).length == 3 - def test_a_phrase_costs_its_entry_its_length_and_its_body(self) -> None: - assert phrase(1, 2, 3).size == PHRASE_TABLE_ENTRY_SIZE + PHRASE_LENGTH_SIZE + 3 + def test_a_phrase_costs_its_entry_its_length_its_count_and_its_body( + self, + ) -> None: + assert phrase(1, 2, 3).size == PHRASE_TABLE_ENTRY_SIZE + PHRASE_LENGTH_SIZE + PHRASE_DEFAULT_SIZE + 3 def test_a_body_reaching_past_a_length_byte_is_refused(self) -> None: with pytest.raises(ValidationError): diff --git a/tests/unit/sampletones_player/compression/dictionary/test_prune.py b/tests/unit/sampletones_player/compression/dictionary/test_prune.py index 479ce6acd..0c2a3709c 100644 --- a/tests/unit/sampletones_player/compression/dictionary/test_prune.py +++ b/tests/unit/sampletones_player/compression/dictionary/test_prune.py @@ -1,21 +1,26 @@ +from collections import defaultdict +from typing import Dict, Final + from sampletones_player.compression.dictionary.prune import prune from sampletones_player.compression.dictionary.table import phrase_table from tests.unit.sampletones_player.compression.dictionary.phrases import phrase +NO_COUNTS: Final[Dict[int, int]] = defaultdict(int) + class TestPruningKeepsWhatPaysForItself: """A phrase earns its place by sparing more bytes than its own entry takes.""" def test_a_phrase_no_token_names_is_dropped(self) -> None: table = phrase_table((phrase(1, 2), phrase(3, 4))) - pruned = prune(table, {0: 2, 1: 0}, {0: 100, 1: 100}) + pruned = prune(table, {0: 2, 1: 0}, {0: 100, 1: 100}, NO_COUNTS) assert pruned.phrases == (phrase(1, 2),) def test_a_phrase_sparing_less_than_its_entry_is_dropped(self) -> None: table = phrase_table((phrase(1, 2),)) - assert prune(table, {0: 1}, {0: phrase(1, 2).size}).phrases == () + assert prune(table, {0: 1}, {0: phrase(1, 2).size}, NO_COUNTS).phrases == () def test_the_phrases_named_most_take_the_cheap_ids(self) -> None: table = phrase_table((phrase(1, 2), phrase(3, 4))) - pruned = prune(table, {0: 1, 1: 9}, {0: 100, 1: 100}) + pruned = prune(table, {0: 1, 1: 9}, {0: 100, 1: 100}, NO_COUNTS) assert pruned.phrases == (phrase(3, 4), phrase(1, 2)) diff --git a/tests/unit/sampletones_player/compression/dictionary/test_table.py b/tests/unit/sampletones_player/compression/dictionary/test_table.py index ef6727a1a..73c85e8d8 100644 --- a/tests/unit/sampletones_player/compression/dictionary/test_table.py +++ b/tests/unit/sampletones_player/compression/dictionary/test_table.py @@ -1,12 +1,18 @@ import pytest from pydantic import ValidationError -from sampletones_player.compression.dictionary.table import PhraseTable, phrase_table +from sampletones_player.compression.dictionary.table import ( + PhraseTable, + phrase_table, +) from sampletones_player.specification.compression import ( MAX_PHRASE_IDS, PHRASE_TABLE_COUNT_SIZE, ) -from tests.unit.sampletones_player.compression.dictionary.phrases import distinct, phrase +from tests.unit.sampletones_player.compression.dictionary.phrases import ( + distinct, + phrase, +) class TestTheTableHoldsWhatATokenCanName: diff --git a/tests/unit/sampletones_player/compression/matches/test_matcher.py b/tests/unit/sampletones_player/compression/matches/test_matcher.py index f35ca0764..9e041ce26 100644 --- a/tests/unit/sampletones_player/compression/matches/test_matcher.py +++ b/tests/unit/sampletones_player/compression/matches/test_matcher.py @@ -4,8 +4,10 @@ from sampletones_player.compression.dictionary.table import phrase_table from sampletones_player.compression.matches.cache import KEY_LENGTH, MatchCache from sampletones_player.compression.matches.index import PlaneIndex -from sampletones_player.compression.matches.match import PhraseMatch -from sampletones_player.compression.matches.matcher import PhraseMatcher +from sampletones_player.compression.matches.matcher import ( + PhraseMatch, + PhraseMatcher, +) from sampletones_player.specification.compression import MAX_PHRASE_TICKS MOTIF: Final[bytes] = bytes((40, 44, 47)) @@ -43,11 +45,15 @@ def test_a_note_outlasting_its_phrase_holds_the_phrase_out(self) -> None: plane = MOTIF + bytes((MOTIF[-1],)) * 4 assert found(plane, (Phrase(body=MOTIF),), 0) == [PhraseMatch(phrase_id=0, ticks=len(plane), transpose=0)] - def test_a_note_cut_short_plays_as_much_of_the_phrase_as_sounded(self) -> None: + def test_a_note_cut_short_plays_as_much_of_the_phrase_as_sounded( + self, + ) -> None: plane = MOTIF[:3] + bytes((99,)) assert found(plane, (Phrase(body=MOTIF),), 0) == [PhraseMatch(phrase_id=0, ticks=3, transpose=0)] - def test_a_note_cut_before_its_shape_is_told_apart_is_offered_nowhere(self) -> None: + def test_a_note_cut_before_its_shape_is_told_apart_is_offered_nowhere( + self, + ) -> None: """A phrase is shortlisted by its first steps, so a shorter start names no phrase.""" plane = MOTIF[:KEY_LENGTH] + bytes((99,)) assert found(plane, (Phrase(body=MOTIF),), 0) == [] diff --git a/tests/unit/sampletones_player/compression/parse/test_boundaries.py b/tests/unit/sampletones_player/compression/parse/test_boundaries.py index d9b7dc1f3..13ad6b0ad 100644 --- a/tests/unit/sampletones_player/compression/parse/test_boundaries.py +++ b/tests/unit/sampletones_player/compression/parse/test_boundaries.py @@ -24,6 +24,8 @@ def test_a_tick_looks_forward_to_the_boundary_ahead_of_it(self) -> None: boundaries = Boundaries.across(TICKS, ENTRIES) assert boundaries.following[0] == 4 - def test_a_tick_past_the_last_boundary_looks_forward_to_the_end_of_the_plane(self) -> None: + def test_a_tick_past_the_last_boundary_looks_forward_to_the_end_of_the_plane( + self, + ) -> None: boundaries = Boundaries.across(TICKS, ENTRIES) assert boundaries.following[4] == TICKS diff --git a/tests/unit/sampletones_player/compression/parse/test_plane.py b/tests/unit/sampletones_player/compression/parse/test_plane.py index ebbadd425..1f8490a63 100644 --- a/tests/unit/sampletones_player/compression/parse/test_plane.py +++ b/tests/unit/sampletones_player/compression/parse/test_plane.py @@ -12,7 +12,10 @@ from sampletones_player.compression.tokens.literal import LiteralToken from sampletones_player.compression.tokens.phrase import PhraseToken from sampletones_player.compression.tokens.types import TokenUnion -from sampletones_player.specification.compression import MAX_HOLD_TICKS, MAX_LITERAL_BYTES +from sampletones_player.specification.compression import ( + MAX_HOLD_TICKS, + MAX_LITERAL_BYTES, +) EVERY_LAYER: Final[CodecOptions] = CodecOptions( holds=True, @@ -63,15 +66,22 @@ def test_the_tokens_cover_every_tick_of_the_plane(self) -> None: parse = parsed(plane) assert sum(token.ticks for token in parse.tokens) == len(plane) - def test_the_cost_of_the_whole_plane_is_the_cost_of_its_tokens(self) -> None: + def test_the_cost_of_the_whole_plane_is_the_cost_of_its_tokens( + self, + ) -> None: parse = parsed(bytes((1, 1, 2, 2, 2, 9))) assert parse.size == sum(token.size for token in parse.tokens) def test_a_run_reaches_the_stream_as_a_hold(self) -> None: parse = parsed(bytes((7,)) * 40) - assert parse.tokens == (LiteralToken(values=bytes((7,))), HoldToken(ticks=39)) + assert parse.tokens == ( + LiteralToken(values=bytes((7,))), + HoldToken(ticks=39), + ) - def test_a_run_longer_than_one_hold_reaches_the_stream_as_several(self) -> None: + def test_a_run_longer_than_one_hold_reaches_the_stream_as_several( + self, + ) -> None: parse = parsed(bytes((7,)) * (2 * MAX_HOLD_TICKS + 1)) assert parse.tokens[1:] == (HoldToken(ticks=MAX_HOLD_TICKS),) * 2 @@ -83,16 +93,20 @@ def test_a_plane_the_codec_finds_nothing_in_spells_itself_out(self) -> None: LiteralToken(values=plane[MAX_LITERAL_BYTES:]), ) - def test_a_figure_the_dictionary_holds_reaches_the_stream_as_a_phrase(self) -> None: + def test_a_figure_the_dictionary_holds_reaches_the_stream_as_a_phrase( + self, + ) -> None: parse = parsed(MOTIF, (Phrase(body=MOTIF),)) - assert parse.tokens == (PhraseToken(phrase_id=0, ticks=len(MOTIF), transpose=0),) + assert parse.tokens == (PhraseToken(phrase_id=0, ticks=len(MOTIF), transpose=0, default=False),) def test_the_same_figure_played_higher_names_the_same_phrase(self) -> None: higher = bytes(value + 7 for value in MOTIF) parse = parsed(higher, (Phrase(body=MOTIF),)) - assert parse.tokens == (PhraseToken(phrase_id=0, ticks=len(MOTIF), transpose=7),) + assert parse.tokens == (PhraseToken(phrase_id=0, ticks=len(MOTIF), transpose=7, default=False),) - def test_a_phrase_the_layer_switches_off_is_spelled_out_instead(self) -> None: + def test_a_phrase_the_layer_switches_off_is_spelled_out_instead( + self, + ) -> None: parse = parsed(MOTIF, (Phrase(body=MOTIF),), options=LITERALS_ONLY) assert parse.tokens == (LiteralToken(values=MOTIF),) @@ -105,7 +119,9 @@ def test_a_token_starts_on_every_boundary(self) -> None: parse = parsed(plane, boundaries=frozenset({0, 7, 13})) assert {0, 7, 13} <= set(starts(parse)) - def test_a_boundary_is_reached_by_a_token_stating_its_own_value(self) -> None: + def test_a_boundary_is_reached_by_a_token_stating_its_own_value( + self, + ) -> None: """A hold plays the value the plane already reached, which a re-entry has yet to state.""" plane = bytes((3,)) * 20 parse = parsed(plane, boundaries=frozenset({0, 7})) diff --git a/tests/unit/sampletones_player/compression/planes/test_channel.py b/tests/unit/sampletones_player/compression/planes/test_channel.py deleted file mode 100644 index 7db19099d..000000000 --- a/tests/unit/sampletones_player/compression/planes/test_channel.py +++ /dev/null @@ -1,21 +0,0 @@ -import pytest -from pydantic import ValidationError - -from sampletones_player.compression.planes.channel import ChannelPlanes - - -class TestAChannelWritesTwoPlanesOfEqualLength: - """The planes are read tick for tick, so a channel states them all across the same ticks.""" - - def test_both_planes_reach_the_ticks_the_channel_covers(self) -> None: - planes = ChannelPlanes(control=bytes(4), value=bytes(4)) - assert planes.ticks == 4 - assert planes.ordered == (planes.control, planes.value) - - def test_planes_covering_different_ticks_are_refused(self) -> None: - with pytest.raises(ValidationError): - ChannelPlanes(control=bytes(3), value=bytes(2)) - - def test_planes_covering_no_tick_are_refused(self) -> None: - with pytest.raises(ValidationError): - ChannelPlanes(control=b"", value=b"") diff --git a/tests/unit/sampletones_player/compression/planes/test_flags.py b/tests/unit/sampletones_player/compression/planes/test_flags.py index f0780ba51..842b6b064 100644 --- a/tests/unit/sampletones_player/compression/planes/test_flags.py +++ b/tests/unit/sampletones_player/compression/planes/test_flags.py @@ -76,6 +76,8 @@ class TestCase(BaseRegularTestCase): def test_the_flags_match(self, test_case: TestCase) -> None: assert note_flags(test_case.indices, test_case.offsets) == test_case.expected - def test_indices_and_offsets_covering_different_ticks_are_refused(self) -> None: + def test_indices_and_offsets_covering_different_ticks_are_refused( + self, + ) -> None: with pytest.raises(ValueError): note_flags((INDEX,), ()) diff --git a/tests/unit/sampletones_player/compression/planes/test_order.py b/tests/unit/sampletones_player/compression/planes/test_order.py index 571c9331c..3df08237d 100644 --- a/tests/unit/sampletones_player/compression/planes/test_order.py +++ b/tests/unit/sampletones_player/compression/planes/test_order.py @@ -1,7 +1,7 @@ import pytest from sampletones_player.compression.planes.order import PlaneOrder -from sampletones_player.specification.compression import PLANE_COUNT +from sampletones_player.specification.planes import PLANE_COUNT def numbered(count: int) -> PlaneOrder: @@ -11,11 +11,15 @@ def numbered(count: int) -> PlaneOrder: class TestThePlanesAreNamedRatherThanNumbered: """The song block writes its planes in one order, and each is reached by its own name.""" - def test_the_planes_take_the_names_the_song_block_writes_them_by(self) -> None: + def test_the_planes_take_the_names_the_song_block_writes_them_by( + self, + ) -> None: planes = numbered(PLANE_COUNT) assert planes.pulse1_control == bytes((0,)) assert planes.noise_value == bytes((PLANE_COUNT - 1,)) - def test_a_run_of_planes_other_than_a_song_block_holds_is_refused(self) -> None: + def test_a_run_of_planes_other_than_a_song_block_holds_is_refused( + self, + ) -> None: with pytest.raises(ValueError): numbered(PLANE_COUNT - 1) diff --git a/tests/unit/sampletones_player/compression/planes/test_rebuild.py b/tests/unit/sampletones_player/compression/planes/test_rebuild.py index 82443b5bb..4f5ddf563 100644 --- a/tests/unit/sampletones_player/compression/planes/test_rebuild.py +++ b/tests/unit/sampletones_player/compression/planes/test_rebuild.py @@ -3,8 +3,16 @@ from sampletones_player.compression.planes.rebuild import streams_from_planes from sampletones_player.compression.planes.separate import planes_from_streams from sampletones_player.registers.streams import ChannelStreams -from tests.suite.player import PLAYER_FULL_VOLUME, noise_tick, player_streams, pulse_tick, triangle_tick -from tests.unit.sampletones_player.compression.planes.conftest import NOISE_PERIOD +from tests.suite.player import ( + PLAYER_FULL_VOLUME, + noise_tick, + player_streams, + pulse_tick, + triangle_tick, +) +from tests.unit.sampletones_player.compression.planes.conftest import ( + NOISE_PERIOD, +) class TestThePlanesReadBackAsTheStreamsTheyCameFrom: diff --git a/tests/unit/sampletones_player/compression/planes/test_separate.py b/tests/unit/sampletones_player/compression/planes/test_separate.py index 5c0ba9fbf..09e83fc99 100644 --- a/tests/unit/sampletones_player/compression/planes/test_separate.py +++ b/tests/unit/sampletones_player/compression/planes/test_separate.py @@ -5,11 +5,20 @@ from sampletones_core.constants.enums import ChannelName from sampletones_player.compression.pitch import PitchTable from sampletones_player.compression.planes.flags import flagged_value -from sampletones_player.compression.planes.separate import channel_planes, planes_from_streams +from sampletones_player.compression.planes.separate import ( + channel_planes, + planes_from_streams, +) from sampletones_player.registers.pulse import PulseRegisters from sampletones_player.registers.streams import ChannelStreams -from sampletones_player.specification.binary import SIGNED_BYTE_LIMIT, unsigned_byte -from sampletones_player.specification.registers import MAX_REGISTER_VALUE, TIMER_HIGH_SHIFT +from sampletones_player.specification.binary import ( + SIGNED_BYTE_LIMIT, + unsigned_byte, +) +from sampletones_player.specification.registers import ( + MAX_REGISTER_VALUE, + TIMER_HIGH_SHIFT, +) from tests.suite.player import PLAYER_FULL_VOLUME, resting_streams from tests.unit.sampletones_player.compression.planes.conftest import ( HIGH_INDEX, @@ -41,7 +50,7 @@ def test_a_tone_channel_names_its_pitch_rather_than_its_divider( pitches: PitchTable, ) -> None: planes = planes_from_streams(sounding_streams, pitches) - assert planes.pulse1.value == bytes((LOW_INDEX, HIGH_INDEX, HIGH_INDEX)) + assert planes.planes.pulse1_value == bytes((LOW_INDEX, HIGH_INDEX, HIGH_INDEX)) def test_the_noise_channel_names_the_period_its_register_takes( self, @@ -49,7 +58,7 @@ def test_the_noise_channel_names_the_period_its_register_takes( pitches: PitchTable, ) -> None: planes = planes_from_streams(sounding_streams, pitches) - assert planes.noise.value == bytes((NOISE_PERIOD,)) * planes.ticks + assert planes.planes.noise_value == bytes((NOISE_PERIOD,)) * planes.ticks def test_a_channel_running_out_early_holds_its_values_through_the_song( self, @@ -58,7 +67,7 @@ def test_a_channel_running_out_early_holds_its_values_through_the_song( ) -> None: planes = planes_from_streams(sounding_streams, pitches) assert planes.ticks == SOUNDING_TICKS - assert planes.pulse2.control == bytes((planes.pulse2.control[0],)) * SOUNDING_TICKS + assert planes.planes.pulse2_control == bytes((planes.planes.pulse2_control[0],)) * SOUNDING_TICKS def test_one_channel_separates_the_same_way_the_song_does( self, @@ -67,7 +76,7 @@ def test_one_channel_separates_the_same_way_the_song_does( ) -> None: planes = planes_from_streams(sounding_streams, pitches) pulse1 = channel_planes(ChannelName.PULSE1, sounding_streams.padded[0], pitches) - assert pulse1 == planes.pulse1 + assert pulse1 == planes.of(ChannelName.PULSE1) class TestABentTickSplitsIntoANoteAndABend: @@ -79,21 +88,21 @@ def test_an_unbent_tick_holds_a_bend_of_nothing( pitches: PitchTable, ) -> None: planes = planes_from_streams(sounding_streams, pitches) - assert planes.pulse1.bend == b"" + assert planes.planes.pulse1_bend == b"" def test_a_bent_tick_names_its_anchor_and_the_steps_from_it(self, pitches: PitchTable) -> None: bends = (2, -3, PAST_HALFWAY) streams = resting_streams([anchored_tick(pitches, LOW_INDEX, bend) for bend in bends]) planes = planes_from_streams(streams, pitches) - assert planes.pulse1.value == bytes(flagged_value(LOW_INDEX, True) for _ in bends) - assert planes.pulse1.bend == bytes(unsigned_byte(bend) for bend in bends) + assert planes.planes.pulse1_value == bytes(flagged_value(LOW_INDEX, True) for _ in bends) + assert planes.planes.pulse1_bend == bytes(unsigned_byte(bend) for bend in bends) def test_a_note_is_flagged_from_its_first_bend_to_its_last(self, pitches: PitchTable) -> None: bends = (0, 3, 0, -3, 0) streams = resting_streams([anchored_tick(pitches, LOW_INDEX, bend) for bend in bends]) planes = planes_from_streams(streams, pitches) - assert planes.pulse1.value == bytes(flagged_value(LOW_INDEX, 0 < tick < 4) for tick in range(len(bends))) - assert planes.pulse1.bend == bytes(unsigned_byte(bend) for bend in bends[1:4]) + assert planes.planes.pulse1_value == bytes(flagged_value(LOW_INDEX, 0 < tick < 4) for tick in range(len(bends))) + assert planes.planes.pulse1_bend == bytes(unsigned_byte(bend) for bend in bends[1:4]) def test_a_divider_past_the_byte_from_its_anchor_is_refused(self, pitches: PitchTable) -> None: streams = resting_streams((anchored_tick(pitches, LOW_INDEX, -SIGNED_BYTE_LIMIT - 1),)) diff --git a/tests/unit/sampletones_player/compression/planes/test_song.py b/tests/unit/sampletones_player/compression/planes/test_song.py index cece618c4..c58cc47ff 100644 --- a/tests/unit/sampletones_player/compression/planes/test_song.py +++ b/tests/unit/sampletones_player/compression/planes/test_song.py @@ -3,72 +3,94 @@ import pytest from pydantic import ValidationError -from sampletones_player.compression.planes.channel import ChannelPlanes, TonePlanes +from sampletones_core.constants.enums import ChannelName from sampletones_player.compression.planes.flags import flagged_value from sampletones_player.compression.planes.order import PlaneOrder from sampletones_player.compression.planes.song import SongPlanes -from sampletones_player.specification.compression import PLANE_COUNT - +from sampletones_player.specification.planes import ( + PLANE_COUNT, + PLANES, + PlaneRole, + plane_index, +) +from tests.suite.player import sounding_planes + +TIMBRE: Final[bytes] = bytes((0x3F, 0x3A)) BEND: Final[bytes] = bytes((0x00, 0xFD)) FLAGGED: Final[bytes] = bytes((flagged_value(3, True), flagged_value(4, True))) -PULSE1_BEND: Final[int] = 2 -BENDS: Final[Tuple[str, ...]] = ("pulse1_bend", "pulse2_bend", "triangle_bend") - - -def song(control: bytes, value: bytes, bend: bytes) -> SongPlanes: - channel = TonePlanes(control=control, value=value, bend=bend) - resting = TonePlanes(control=bytes(len(control)), value=bytes(len(value)), bend=b"") - silent = ChannelPlanes(control=bytes(len(control)), value=bytes(len(value))) - return SongPlanes(pulse1=channel, pulse2=resting, triangle=resting, noise=silent) +PULSE1_BEND: Final[int] = plane_index(ChannelName.PULSE1, PlaneRole.BEND) +BENDS: Final[Tuple[str, ...]] = tuple(plane.name for plane in PLANES if plane.spans_flagged_ticks) -class TestASongGathersItsChannelsPlanes: +class TestASongGathersEveryPlaneItsBlockWrites: """Every plane advances beside the rest, so the song states them all under one length.""" def test_a_song_carries_the_planes_a_block_writes(self) -> None: - assert len(song(bytes((1, 2)), FLAGGED, BEND).planes) == PLANE_COUNT + assert len(sounding_planes(TIMBRE, FLAGGED, BEND).planes) == PLANE_COUNT - def test_the_planes_read_back_under_the_channels_that_write_them(self) -> None: - planes = song(bytes((1, 2)), FLAGGED, BEND) - assert SongPlanes.from_order(planes.planes) == planes + def test_a_plane_is_reached_by_the_name_the_block_writes_it_under( + self, + ) -> None: + planes = sounding_planes(TIMBRE, FLAGGED, BEND).planes - def test_a_tone_channels_bend_reaches_the_block_beside_its_pitch(self) -> None: - planes = song(bytes((1, 2)), FLAGGED, BEND).planes assert planes.pulse1_bend == BEND - def test_a_song_lasts_the_ticks_its_channels_cover(self) -> None: - assert song(bytes((1, 2)), FLAGGED, BEND).ticks == 2 + def test_a_channel_answers_with_the_planes_it_writes(self) -> None: + planes = sounding_planes(TIMBRE, FLAGGED, BEND) + + assert planes.of(ChannelName.PULSE1) == (TIMBRE, FLAGGED, BEND) + + def test_a_song_lasts_the_ticks_its_planes_cover(self) -> None: + assert sounding_planes(TIMBRE, FLAGGED, BEND).ticks == 2 - def test_channels_covering_different_ticks_are_refused(self) -> None: - short = TonePlanes(control=bytes(1), value=bytes(1), bend=b"") - long = TonePlanes(control=bytes(2), value=bytes(2), bend=b"") + def test_planes_covering_different_ticks_are_refused(self) -> None: with pytest.raises(ValidationError): SongPlanes( - pulse1=short, - pulse2=long, - triangle=short, - noise=ChannelPlanes(control=bytes(1), value=bytes(1)), + planes=PlaneOrder.across(bytes((plane.seeded,)) * index for index, plane in enumerate(PLANES, start=1)) ) + def test_planes_covering_no_tick_are_refused(self) -> None: + with pytest.raises(ValidationError): + SongPlanes(planes=PlaneOrder.across(b"" for _ in range(PLANE_COUNT))) + class TestABendPlaneCoversTheFlaggedTicks: """A bend plane holds a value for each tick its value plane flags, and for those alone.""" def test_a_bend_for_every_flagged_tick_is_taken(self) -> None: value = bytes((flagged_value(3, False), flagged_value(3, True))) - assert TonePlanes(control=bytes(2), value=value, bend=BEND[:1]).bend == BEND[:1] - def test_a_bend_plane_holding_more_than_the_flags_ask_for_is_refused(self) -> None: - with pytest.raises(ValidationError): - TonePlanes(control=bytes(2), value=bytes((3, 4)), bend=BEND) + assert sounding_planes(TIMBRE, value, BEND[:1]).planes.pulse1_bend == BEND[:1] - def test_a_bend_plane_holding_fewer_than_the_flags_ask_for_is_refused(self) -> None: + def test_a_bend_plane_holding_more_than_the_flags_ask_for_is_refused( + self, + ) -> None: with pytest.raises(ValidationError): - TonePlanes(control=bytes(2), value=FLAGGED, bend=BEND[:1]) + sounding_planes(TIMBRE, bytes((3, 4)), BEND) - def test_a_song_returning_to_a_tick_re_enters_the_bend_plane_past_the_flags_before_it(self) -> None: - value = bytes((flagged_value(3, True), flagged_value(3, False), flagged_value(3, True))) - planes = song(bytes(3), value, BEND) - assert [planes.positions(tick)[PULSE1_BEND] for tick in range(4)] == [0, 1, 1, 2] + def test_a_bend_plane_holding_fewer_than_the_flags_ask_for_is_refused( + self, + ) -> None: + with pytest.raises(ValidationError): + sounding_planes(TIMBRE, FLAGGED, BEND[:1]) + + def test_a_song_returning_to_a_tick_re_enters_the_bend_plane_past_the_flags_before_it( + self, + ) -> None: + value = bytes( + ( + flagged_value(3, True), + flagged_value(3, False), + flagged_value(3, True), + ) + ) + planes = sounding_planes(TIMBRE + bytes((0x30,)), value, BEND) + + assert [planes.positions(tick)[PULSE1_BEND] for tick in range(4)] == [ + 0, + 1, + 1, + 2, + ] dense = [position for name, position in zip(PlaneOrder.names(), planes.positions(2)) if name not in BENDS] assert dense == [2] * len(dense) diff --git a/tests/unit/sampletones_player/compression/test_admit.py b/tests/unit/sampletones_player/compression/test_admit.py index 9a6d360bb..ee0eec0d8 100644 --- a/tests/unit/sampletones_player/compression/test_admit.py +++ b/tests/unit/sampletones_player/compression/test_admit.py @@ -123,4 +123,8 @@ def test_the_run_says_the_table_could_not_take_them_all( def crowded_seeds() -> Tuple[Phrase, ...]: """More shapes than a token can name, two of which the plane actually plays.""" - return (*unplayed_seeds(), Phrase(body=LEANED_ON), Phrase(body=PLAYED_SELDOM)) + return ( + *unplayed_seeds(), + Phrase(body=LEANED_ON), + Phrase(body=PLAYED_SELDOM), + ) diff --git a/tests/unit/sampletones_player/compression/test_budget.py b/tests/unit/sampletones_player/compression/test_budget.py index e39a02db0..d4d0f7225 100644 --- a/tests/unit/sampletones_player/compression/test_budget.py +++ b/tests/unit/sampletones_player/compression/test_budget.py @@ -4,7 +4,11 @@ import pytest from pydantic import ValidationError -from sampletones_player.compression.budget import DEFAULT_SEARCH_BUDGET, SearchBudget, shares +from sampletones_player.compression.budget import ( + DEFAULT_SEARCH_BUDGET, + SearchBudget, + shares, +) from tests.suite.base import BaseTestSuite from tests.suite.case import BaseRegularTestCase @@ -19,11 +23,36 @@ class TestCase(BaseRegularTestCase): expected: Tuple[int, ...] test_cases = ( - TestCase(label="every_demand_fits", demands=(5, 5, 5), total=30, expected=(5, 5, 5)), - TestCase(label="a_small_demand_leaves_its_surplus", demands=(100, 1, 100), total=41, expected=(20, 1, 20)), - TestCase(label="a_zero_demand_takes_nothing", demands=(0, 7), total=3, expected=(0, 3)), - TestCase(label="equal_demands_split_evenly", demands=(9, 9), total=8, expected=(4, 4)), - TestCase(label="a_total_short_of_one_each", demands=(9, 9, 9), total=2, expected=(0, 1, 1)), + TestCase( + label="every_demand_fits", + demands=(5, 5, 5), + total=30, + expected=(5, 5, 5), + ), + TestCase( + label="a_small_demand_leaves_its_surplus", + demands=(100, 1, 100), + total=41, + expected=(20, 1, 20), + ), + TestCase( + label="a_zero_demand_takes_nothing", + demands=(0, 7), + total=3, + expected=(0, 3), + ), + TestCase( + label="equal_demands_split_evenly", + demands=(9, 9), + total=8, + expected=(4, 4), + ), + TestCase( + label="a_total_short_of_one_each", + demands=(9, 9, 9), + total=2, + expected=(0, 1, 1), + ), TestCase(label="no_claimants", demands=(), total=10, expected=()), ) diff --git a/tests/unit/sampletones_player/compression/test_decode.py b/tests/unit/sampletones_player/compression/test_decode.py index 5ecb01256..14ea7fdbf 100644 --- a/tests/unit/sampletones_player/compression/test_decode.py +++ b/tests/unit/sampletones_player/compression/test_decode.py @@ -1,11 +1,22 @@ from typing import Final +from sampletones_core.constants.enums import ChannelName from sampletones_player.compression.decode import decode_plane from sampletones_player.compression.dictionary.phrase import Phrase from sampletones_player.compression.dictionary.table import phrase_table -from sampletones_player.specification.compression import PHRASE_ID_ESCAPE, TokenTag +from sampletones_player.specification.compression import ( + PHRASE_ID_ESCAPE, + TokenTag, +) +from sampletones_player.specification.planes import ( + PLANES, + Plane, + PlaneRole, + plane_index, +) MOTIF: Final[bytes] = bytes((40, 44, 47)) +SPELLED_PLANE: Final[Plane] = PLANES[plane_index(ChannelName.PULSE1, PlaneRole.VALUE)] DICTIONARY = phrase_table((Phrase(body=MOTIF),)) @@ -14,40 +25,56 @@ class TestWhatTheDriverReadsFromAStream: def test_a_literal_writes_the_values_that_follow_it(self) -> None: stream = bytes((TokenTag.LITERAL | 2, 5, 6, 7)) - assert decode_plane(stream, DICTIONARY, 3) == bytes((5, 6, 7)) + assert decode_plane(stream, DICTIONARY, SPELLED_PLANE, 3) == bytes((5, 6, 7)) def test_a_hold_writes_the_value_the_plane_reached(self) -> None: stream = bytes((TokenTag.LITERAL | 0, 9, TokenTag.HOLD | 2)) - assert decode_plane(stream, DICTIONARY, 4) == bytes((9, 9, 9, 9)) + assert decode_plane(stream, DICTIONARY, SPELLED_PLANE, 4) == bytes((9, 9, 9, 9)) def test_a_phrase_writes_the_body_the_table_holds(self) -> None: stream = bytes((TokenTag.PHRASE | 0, len(MOTIF) - 1)) - assert decode_plane(stream, DICTIONARY, len(MOTIF)) == MOTIF + assert decode_plane(stream, DICTIONARY, SPELLED_PLANE, len(MOTIF)) == MOTIF - def test_a_phrase_running_past_its_body_holds_the_value_it_ended_on(self) -> None: + def test_a_phrase_running_past_its_body_holds_the_value_it_ended_on( + self, + ) -> None: stream = bytes((TokenTag.PHRASE | 0, len(MOTIF) + 1)) - assert decode_plane(stream, DICTIONARY, len(MOTIF) + 2) == MOTIF + bytes((MOTIF[-1],)) * 2 + assert decode_plane(stream, DICTIONARY, SPELLED_PLANE, len(MOTIF) + 2) == MOTIF + bytes((MOTIF[-1],)) * 2 - def test_a_phrase_cut_short_writes_as_much_of_the_body_as_sounded(self) -> None: + def test_a_phrase_cut_short_writes_as_much_of_the_body_as_sounded( + self, + ) -> None: stream = bytes((TokenTag.PHRASE | 0, 1)) - assert decode_plane(stream, DICTIONARY, 2) == MOTIF[:2] + assert decode_plane(stream, DICTIONARY, SPELLED_PLANE, 2) == MOTIF[:2] def test_a_shifted_phrase_writes_the_body_moved_by_the_shift(self) -> None: stream = bytes((TokenTag.TRANSPOSED_PHRASE | 0, len(MOTIF) - 1, 5)) - assert decode_plane(stream, DICTIONARY, len(MOTIF)) == bytes(value + 5 for value in MOTIF) + assert decode_plane(stream, DICTIONARY, SPELLED_PLANE, len(MOTIF)) == bytes(value + 5 for value in MOTIF) - def test_a_shift_walks_the_byte_around_where_it_reaches_past_one(self) -> None: + def test_a_shift_walks_the_byte_around_where_it_reaches_past_one( + self, + ) -> None: """The driver adds the shift to a byte, so a fall reaches it as the byte that wraps to it.""" stream = bytes((TokenTag.TRANSPOSED_PHRASE | 0, 0, 0xFD)) - assert decode_plane(stream, DICTIONARY, 1) == bytes((MOTIF[0] - 3,)) + assert decode_plane(stream, DICTIONARY, SPELLED_PLANE, 1) == bytes((MOTIF[0] - 3,)) - def test_an_escaped_phrase_names_its_id_in_the_byte_that_follows(self) -> None: + def test_an_escaped_phrase_names_its_id_in_the_byte_that_follows( + self, + ) -> None: table = phrase_table( tuple(Phrase(body=bytes((value, value))) for value in range(PHRASE_ID_ESCAPE)) + (Phrase(body=MOTIF),) ) - stream = bytes((TokenTag.PHRASE | PHRASE_ID_ESCAPE, PHRASE_ID_ESCAPE, len(MOTIF) - 1)) - assert decode_plane(stream, table, len(MOTIF)) == MOTIF + stream = bytes( + ( + TokenTag.PHRASE | PHRASE_ID_ESCAPE, + PHRASE_ID_ESCAPE, + len(MOTIF) - 1, + ) + ) + assert decode_plane(stream, table, SPELLED_PLANE, len(MOTIF)) == MOTIF - def test_a_token_reaching_past_the_song_stops_where_the_song_does(self) -> None: + def test_a_token_reaching_past_the_song_stops_where_the_song_does( + self, + ) -> None: stream = bytes((TokenTag.LITERAL | 0, 4, TokenTag.HOLD | 63)) - assert decode_plane(stream, DICTIONARY, 3) == bytes((4, 4, 4)) + assert decode_plane(stream, DICTIONARY, SPELLED_PLANE, 3) == bytes((4, 4, 4)) diff --git a/tests/unit/sampletones_player/compression/test_encode.py b/tests/unit/sampletones_player/compression/test_encode.py index c26c2bded..866055739 100644 --- a/tests/unit/sampletones_player/compression/test_encode.py +++ b/tests/unit/sampletones_player/compression/test_encode.py @@ -6,7 +6,6 @@ from sampletones_player.compression.dictionary.phrase import Phrase from sampletones_player.compression.encode import emit, encode_planes from sampletones_player.compression.options import CodecOptions -from sampletones_player.compression.planes.channel import ChannelPlanes, TonePlanes from sampletones_player.compression.planes.song import SongPlanes from sampletones_player.compression.progress.report import CodecProgress from sampletones_player.compression.tokens.hold import HoldToken @@ -18,6 +17,7 @@ TokenTag, ) from sampletones_shared.exceptions import OperationCanceled +from tests.suite.player import sounding_planes from tests.suite.progress import FIRST_REPORT, RecordingReporter EVERY_LAYER: Final[CodecOptions] = CodecOptions( @@ -39,11 +39,7 @@ def song_planes(control: bytes, value: bytes) -> SongPlanes: - unbent = bytes(len(control)) - channel = TonePlanes(control=control, value=value, bend=b"") - resting = TonePlanes(control=unbent, value=bytes(len(value)), bend=b"") - silent = ChannelPlanes(control=unbent, value=bytes(len(value))) - return SongPlanes(pulse1=channel, pulse2=resting, triangle=resting, noise=silent) + return sounding_planes(control, value, b"") class TestWhatATokenLooksLikeOnTheBus: @@ -56,19 +52,25 @@ def test_a_literal_carries_its_length_inside_its_opcode(self) -> None: assert emit((LiteralToken(values=bytes((0x10, 0x20))),)) == bytes((TokenTag.LITERAL | 1, 0x10, 0x20)) def test_a_phrase_carries_a_cheap_id_inside_its_opcode(self) -> None: - token = PhraseToken(phrase_id=2, ticks=5, transpose=0) + token = PhraseToken(phrase_id=2, ticks=5, transpose=0, default=False) assert emit((token,)) == bytes((TokenTag.PHRASE | 2, 4)) def test_a_shifted_phrase_states_the_shift_after_the_count(self) -> None: - token = PhraseToken(phrase_id=2, ticks=5, transpose=0xFD) + token = PhraseToken(phrase_id=2, ticks=5, transpose=0xFD, default=False) assert emit((token,)) == bytes((TokenTag.TRANSPOSED_PHRASE | 2, 4, 0xFD)) - def test_a_phrase_beyond_the_cheap_ids_names_itself_in_the_byte_that_follows(self) -> None: - token = PhraseToken(phrase_id=200, ticks=MAX_PHRASE_TICKS, transpose=0) + def test_a_phrase_beyond_the_cheap_ids_names_itself_in_the_byte_that_follows( + self, + ) -> None: + token = PhraseToken(phrase_id=200, ticks=MAX_PHRASE_TICKS, transpose=0, default=False) assert emit((token,)) == bytes((TokenTag.PHRASE | PHRASE_ID_ESCAPE, 200, MAX_PHRASE_TICKS - 1)) def test_a_stream_takes_the_bytes_the_parse_counted(self) -> None: - tokens = (LiteralToken(values=MOTIF), HoldToken(ticks=4), PhraseToken(phrase_id=1, ticks=2, transpose=3)) + tokens = ( + LiteralToken(values=MOTIF), + HoldToken(ticks=4), + PhraseToken(phrase_id=1, ticks=2, transpose=3, default=False), + ) assert len(emit(tokens)) == sum(token.size for token in tokens) @@ -111,16 +113,21 @@ def test_a_seed_the_song_never_leans_on_leaves_the_dictionary(self) -> None: def test_the_figure_played_most_takes_the_cheapest_id(self) -> None: planes = song_planes(bytes((0x30,)) * (len(MOTIF) * REPEATS), MOTIF * REPEATS) - seeds: Tuple[Phrase, ...] = (Phrase(body=bytes((0x30,)) * 8), Phrase(body=MOTIF)) + seeds: Tuple[Phrase, ...] = ( + Phrase(body=bytes((0x30,)) * 8), + Phrase(body=MOTIF), + ) compressed = encode_planes( planes, seeds, options=SEEDED, boundaries=NO_BOUNDARIES, ) - assert compressed.phrases[0] == Phrase(body=MOTIF) + assert compressed.phrases[0].body == MOTIF - def test_a_song_re_entered_at_a_boundary_still_plays_back_whole(self) -> None: + def test_a_song_re_entered_at_a_boundary_still_plays_back_whole( + self, + ) -> None: planes = song_planes(TIMBRE * REPEATS, MOTIF * REPEATS) compressed = encode_planes( planes, diff --git a/tests/unit/sampletones_player/compression/test_entries.py b/tests/unit/sampletones_player/compression/test_entries.py index 9e31676c1..a901a0d72 100644 --- a/tests/unit/sampletones_player/compression/test_entries.py +++ b/tests/unit/sampletones_player/compression/test_entries.py @@ -2,6 +2,10 @@ import pytest +from sampletones_player.compression.dictionary.table import ( + PhraseTable, + phrase_table, +) from sampletones_player.compression.encode import emit from sampletones_player.compression.entries import stream_entry from sampletones_player.compression.tokens.hold import HoldToken @@ -9,6 +13,7 @@ from sampletones_player.compression.tokens.phrase import PhraseToken FIRST_TICK: Final[int] = 0 +DICTIONARY: Final[PhraseTable] = phrase_table(()) PHRASE_ID: Final[int] = 2 SHIFT: Final[int] = 7 @@ -18,25 +23,31 @@ class TestWhereAStreamIsReEntered: def test_the_first_tick_is_the_streams_own_first_byte(self) -> None: stream = emit([LiteralToken(values=b"\x01\x02"), HoldToken(ticks=3)]) - assert stream_entry(stream, FIRST_TICK) == 0 + assert stream_entry(stream, FIRST_TICK, DICTIONARY) == 0 def test_a_tick_a_token_starts_answers_with_that_tokens_byte(self) -> None: literal = LiteralToken(values=b"\x01\x02") - stream = emit([literal, HoldToken(ticks=3), PhraseToken(phrase_id=PHRASE_ID, ticks=4, transpose=0)]) - assert stream_entry(stream, literal.ticks) == literal.size + stream = emit( + [ + literal, + HoldToken(ticks=3), + PhraseToken(phrase_id=PHRASE_ID, ticks=4, transpose=0, default=False), + ] + ) + assert stream_entry(stream, literal.ticks, DICTIONARY) == literal.size def test_a_tick_further_in_walks_past_every_token_before_it(self) -> None: tokens = [ LiteralToken(values=b"\x01\x02"), HoldToken(ticks=3), - PhraseToken(phrase_id=PHRASE_ID, ticks=4, transpose=SHIFT), + PhraseToken(phrase_id=PHRASE_ID, ticks=4, transpose=SHIFT, default=False), ] stream = emit(tokens) covered = sum(token.ticks for token in tokens[:2]) - assert stream_entry(stream, covered) == sum(token.size for token in tokens[:2]) + assert stream_entry(stream, covered, DICTIONARY) == sum(token.size for token in tokens[:2]) def test_a_tick_a_token_spans_is_refused(self) -> None: """A stream re-entered mid-token would leave the driver reading operands as opcodes.""" stream = emit([HoldToken(ticks=8)]) with pytest.raises(ValueError, match="spans position"): - stream_entry(stream, 3) + stream_entry(stream, 3, DICTIONARY) diff --git a/tests/unit/sampletones_player/compression/test_scheme.py b/tests/unit/sampletones_player/compression/test_scheme.py index 53ce37a8c..644d0b842 100644 --- a/tests/unit/sampletones_player/compression/test_scheme.py +++ b/tests/unit/sampletones_player/compression/test_scheme.py @@ -3,11 +3,19 @@ import pytest from sampletones_player.compression.options import EVERY_LAYER, CodecOptions -from sampletones_player.compression.scheme import CompressionScheme, offered_schemes +from sampletones_player.compression.scheme import ( + CompressionScheme, + offered_schemes, +) def layers(options: CodecOptions) -> Tuple[bool, ...]: - return (options.holds, options.phrases, options.transposition, options.search) + return ( + options.holds, + options.phrases, + options.transposition, + options.search, + ) class TestCompressionScheme: @@ -40,7 +48,9 @@ class TestOfferedSchemes: def test_a_seeded_song_is_offered_every_scheme(self) -> None: assert offered_schemes(seeded=True) == tuple(CompressionScheme) - def test_a_song_seeding_nothing_is_offered_every_scheme_but_the_instruments(self) -> None: + def test_a_song_seeding_nothing_is_offered_every_scheme_but_the_instruments( + self, + ) -> None: assert offered_schemes(seeded=False) == tuple( scheme for scheme in CompressionScheme if scheme != CompressionScheme.INSTRUMENTS ) diff --git a/tests/unit/sampletones_player/compression/test_search.py b/tests/unit/sampletones_player/compression/test_search.py index 8f1970da4..300b322f0 100644 --- a/tests/unit/sampletones_player/compression/test_search.py +++ b/tests/unit/sampletones_player/compression/test_search.py @@ -4,7 +4,10 @@ import pytest from sampletones_player.compression.budget import SearchBudget -from sampletones_player.compression.dictionary.table import PhraseTable, phrase_table +from sampletones_player.compression.dictionary.table import ( + PhraseTable, + phrase_table, +) from sampletones_player.compression.matches.cache import MatchCache from sampletones_player.compression.matches.index import PlaneIndex from sampletones_player.compression.options import EVERY_LAYER @@ -63,9 +66,21 @@ class TestTheRoundsBoundWhatTheSearchEarns: """Each round adds at most one phrase, so the rounds cap the dictionary the search fills.""" def test_no_rounds_leaves_the_table_as_seeded(self) -> None: - table = _searched(SearchBudget(candidate_entries=NARROW_ENTRIES, rounds=0, confirmed_candidates=3)) + table = _searched( + SearchBudget( + candidate_entries=NARROW_ENTRIES, + rounds=0, + confirmed_candidates=3, + ) + ) assert len(table) == 0 def test_one_round_adds_one_phrase(self) -> None: - table = _searched(SearchBudget(candidate_entries=NARROW_ENTRIES, rounds=1, confirmed_candidates=3)) + table = _searched( + SearchBudget( + candidate_entries=NARROW_ENTRIES, + rounds=1, + confirmed_candidates=3, + ) + ) assert len(table) == 1 diff --git a/tests/unit/sampletones_player/compression/test_seeds.py b/tests/unit/sampletones_player/compression/test_seeds.py index 1ff792f1a..f6d074ebf 100644 --- a/tests/unit/sampletones_player/compression/test_seeds.py +++ b/tests/unit/sampletones_player/compression/test_seeds.py @@ -1,4 +1,4 @@ -from typing import Final, Tuple +from typing import Final, FrozenSet, Tuple import pytest @@ -9,13 +9,18 @@ from sampletones_core.project.voices.instrument import Instrument from sampletones_core.timers.utils import get_timer_table from sampletones_player.compression.pitch import PitchTable -from sampletones_player.compression.planes.channel import ChannelPlanes from sampletones_player.compression.planes.separate import channel_planes +from sampletones_player.compression.planes.symbols import pack_plane from sampletones_player.compression.seeds import phrases_from_project from sampletones_player.registers.channel import channel_registers from sampletones_player.specification.binary import unsigned_byte +from sampletones_player.specification.planes import PLANES from sampletones_shared.music import Tuning -from tests.suite.performance import make_pulse_reconstruction, project_with_instrument, project_with_sample +from tests.suite.performance import ( + make_pulse_reconstruction, + project_with_instrument, + project_with_sample, +) from tests.suite.player import PLAYER_FULL_VOLUME, PLAYER_REFERENCE_PITCH TUNING: Final[Tuning] = Tuning() @@ -33,7 +38,7 @@ def project() -> Project: @pytest.fixture -def slice_planes(project: Project) -> ChannelPlanes: +def slice_planes(project: Project) -> Tuple[bytes, ...]: """The planes the project's own slice writes on the channel it plays.""" instructions = project.voices[0].reconstruction.get_channel_instructions(ChannelName.PULSE1) registers = channel_registers( @@ -48,24 +53,31 @@ def _offered(project: Project) -> Tuple[bytes, ...]: return tuple(phrase.body for phrase in phrases_from_project(project, TUNING, ALL_CHANNELS)) +NO_BOUNDARIES: Final[FrozenSet[int]] = frozenset() + + class TestTheInstrumentsSeedTheDictionary: """A song plays sample slices at rows, so the shapes its planes repeat are the slices.""" def test_the_phrases_are_the_planes_the_slice_turns_over( self, project: Project, - slice_planes: ChannelPlanes, + slice_planes: Tuple[bytes, ...], ) -> None: - turning = tuple(plane for plane in slice_planes.ordered if len(set(plane)) > 1) + turning = tuple( + pack_plane(plane, PLANES[index].form, boundaries=NO_BOUNDARIES) + for index, plane in enumerate(slice_planes) + if len(set(plane)) > 1 + ) assert _offered(project) == turning def test_a_plane_holding_one_value_offers_the_dictionary_nothing( self, project: Project, - slice_planes: ChannelPlanes, + slice_planes: Tuple[bytes, ...], ) -> None: """A hold covers such a plane more cheaply than any phrase naming it could.""" - held = tuple(plane for plane in slice_planes.ordered if len(set(plane)) == 1) + held = tuple(plane for plane in slice_planes if len(set(plane)) == 1) assert held assert not set(held) & set(_offered(project)) diff --git a/tests/unit/sampletones_player/compression/test_song.py b/tests/unit/sampletones_player/compression/test_song.py index f59dd9e2c..c6a4dc87d 100644 --- a/tests/unit/sampletones_player/compression/test_song.py +++ b/tests/unit/sampletones_player/compression/test_song.py @@ -50,7 +50,9 @@ def test_every_tick_writes_the_registers_it_was_given(self) -> None: streams.at(tick) for tick in range(streams.ticks) ] - def test_a_channel_running_out_early_is_carried_to_the_songs_length(self) -> None: + def test_a_channel_running_out_early_is_carried_to_the_songs_length( + self, + ) -> None: streams = resting_streams((SOUNDING, OCTAVE_UP, RESTING)) rebuilt = played(streams, None) assert len(rebuilt.noise) == streams.ticks diff --git a/tests/unit/sampletones_player/compression/tokens/test_hold.py b/tests/unit/sampletones_player/compression/tokens/test_hold.py index 4e9a86220..af7f0b58d 100644 --- a/tests/unit/sampletones_player/compression/tokens/test_hold.py +++ b/tests/unit/sampletones_player/compression/tokens/test_hold.py @@ -1,5 +1,8 @@ from sampletones_player.compression.tokens.hold import HoldToken -from sampletones_player.specification.compression import MAX_HOLD_TICKS, OPCODE_SIZE +from sampletones_player.specification.compression import ( + MAX_HOLD_TICKS, + OPCODE_SIZE, +) class TestWhatAHoldCosts: diff --git a/tests/unit/sampletones_player/compression/tokens/test_literal.py b/tests/unit/sampletones_player/compression/tokens/test_literal.py index d78b237bf..ac708e21d 100644 --- a/tests/unit/sampletones_player/compression/tokens/test_literal.py +++ b/tests/unit/sampletones_player/compression/tokens/test_literal.py @@ -11,6 +11,8 @@ class TestWhatALiteralCosts: def test_a_literal_covers_a_tick_for_every_value_it_states(self) -> None: assert LiteralToken(values=bytes(SPELLED_OUT)).ticks == SPELLED_OUT - def test_a_literal_costs_its_opcode_and_the_values_it_spells_out(self) -> None: + def test_a_literal_costs_its_opcode_and_the_values_it_spells_out( + self, + ) -> None: token = LiteralToken(values=bytes(SPELLED_OUT)) assert token.size == literal_size(SPELLED_OUT) == OPCODE_SIZE + SPELLED_OUT diff --git a/tests/unit/sampletones_player/compression/tokens/test_phrase.py b/tests/unit/sampletones_player/compression/tokens/test_phrase.py index b60c280d8..6048c6cf3 100644 --- a/tests/unit/sampletones_player/compression/tokens/test_phrase.py +++ b/tests/unit/sampletones_player/compression/tokens/test_phrase.py @@ -51,5 +51,10 @@ def label(self) -> str: @pytest.mark.parametrize("test_case", test_cases, ids=lambda test_case: test_case.label) def test_a_phrase_token_pays_for_the_id_and_the_shift_it_names(self, test_case: TestCase) -> None: - token = PhraseToken(phrase_id=test_case.phrase_id, ticks=1, transpose=test_case.transpose) - assert token.size == test_case.expected == phrase_size(test_case.phrase_id, test_case.transpose) + token = PhraseToken( + phrase_id=test_case.phrase_id, + ticks=1, + transpose=test_case.transpose, + default=False, + ) + assert token.size == test_case.expected == phrase_size(test_case.phrase_id, test_case.transpose, default=False) diff --git a/tests/unit/sampletones_player/compression/tokens/test_span.py b/tests/unit/sampletones_player/compression/tokens/test_span.py index 6420af0ef..3f4a5d94f 100644 --- a/tests/unit/sampletones_player/compression/tokens/test_span.py +++ b/tests/unit/sampletones_player/compression/tokens/test_span.py @@ -3,6 +3,11 @@ import pytest +from sampletones_player.compression.dictionary.phrase import Phrase +from sampletones_player.compression.dictionary.table import ( + PhraseTable, + phrase_table, +) from sampletones_player.compression.encode import emit from sampletones_player.compression.tokens.hold import HoldToken from sampletones_player.compression.tokens.literal import LiteralToken @@ -21,6 +26,10 @@ CHEAP_ID: Final[int] = 1 SHIFT: Final[int] = 5 PHRASE_PLAY_TICKS: Final[int] = 10 +CARRIED_TICKS: Final[int] = 6 +DICTIONARY: Final[PhraseTable] = phrase_table( + tuple(Phrase(body=bytes((value,)), default=CARRIED_TICKS) for value in range(PHRASE_ID_ESCAPE + 2)) +) class TestWhatAWrittenTokenTakesAndCovers(BaseTestSuite): @@ -37,7 +46,11 @@ def label(self) -> str: return self.name test_cases = ( - TestCase(name="hold", token=HoldToken(ticks=7), expected=TokenSpan(size=1, ticks=7)), + TestCase( + name="hold", + token=HoldToken(ticks=7), + expected=TokenSpan(size=1, ticks=7), + ), TestCase( name="hold-longest", token=HoldToken(ticks=MAX_HOLD_TICKS), @@ -50,17 +63,32 @@ def label(self) -> str: ), TestCase( name="phrase", - token=PhraseToken(phrase_id=CHEAP_ID, ticks=PHRASE_PLAY_TICKS, transpose=0), + token=PhraseToken( + phrase_id=CHEAP_ID, + ticks=PHRASE_PLAY_TICKS, + transpose=0, + default=False, + ), expected=TokenSpan(size=2, ticks=PHRASE_PLAY_TICKS), ), TestCase( name="phrase-shifted", - token=PhraseToken(phrase_id=CHEAP_ID, ticks=PHRASE_PLAY_TICKS, transpose=SHIFT), + token=PhraseToken( + phrase_id=CHEAP_ID, + ticks=PHRASE_PLAY_TICKS, + transpose=SHIFT, + default=False, + ), expected=TokenSpan(size=3, ticks=PHRASE_PLAY_TICKS), ), TestCase( name="phrase-escaped", - token=PhraseToken(phrase_id=PHRASE_ID_ESCAPE, ticks=PHRASE_PLAY_TICKS, transpose=0), + token=PhraseToken( + phrase_id=PHRASE_ID_ESCAPE, + ticks=PHRASE_PLAY_TICKS, + transpose=0, + default=False, + ), expected=TokenSpan(size=3, ticks=PHRASE_PLAY_TICKS), ), TestCase( @@ -69,30 +97,36 @@ def label(self) -> str: phrase_id=PHRASE_ID_ESCAPE, ticks=PHRASE_PLAY_TICKS, transpose=SHIFT, + default=False, ), expected=TokenSpan(size=4, ticks=PHRASE_PLAY_TICKS), ), TestCase( name="phrase-longest", - token=PhraseToken(phrase_id=CHEAP_ID, ticks=MAX_PHRASE_TICKS, transpose=0), + token=PhraseToken( + phrase_id=CHEAP_ID, + ticks=MAX_PHRASE_TICKS, + transpose=0, + default=False, + ), expected=TokenSpan(size=2, ticks=MAX_PHRASE_TICKS), ), ) @pytest.mark.parametrize("test_case", test_cases, ids=lambda test_case: test_case.label) def test_the_span_states_what_the_token_takes(self, test_case: TestCase) -> None: - span = token_span(emit([test_case.token]), FIRST_TOKEN) + span = token_span(emit([test_case.token]), FIRST_TOKEN, DICTIONARY) assert span.size == test_case.expected.size @pytest.mark.parametrize("test_case", test_cases, ids=lambda test_case: test_case.label) def test_the_span_states_what_the_token_covers(self, test_case: TestCase) -> None: - span = token_span(emit([test_case.token]), FIRST_TOKEN) + span = token_span(emit([test_case.token]), FIRST_TOKEN, DICTIONARY) assert span.ticks == test_case.expected.ticks @pytest.mark.parametrize("test_case", test_cases, ids=lambda test_case: test_case.label) def test_the_span_reaches_the_byte_the_next_token_begins_at(self, test_case: TestCase) -> None: stream = emit([test_case.token, HoldToken(ticks=1)]) - assert token_span(stream, FIRST_TOKEN).size == len(stream) - 1 + assert token_span(stream, FIRST_TOKEN, DICTIONARY).size == len(stream) - 1 class TestWalkingAStream: @@ -102,12 +136,12 @@ def test_the_sizes_reach_the_streams_last_byte(self) -> None: tokens = ( LiteralToken(values=b"\x01\x02"), HoldToken(ticks=3), - PhraseToken(phrase_id=CHEAP_ID, ticks=4, transpose=SHIFT), + PhraseToken(phrase_id=CHEAP_ID, ticks=4, transpose=SHIFT, default=False), ) stream = emit(tokens) position = 0 for _ in tokens: - position += token_span(stream, position).size + position += token_span(stream, position, DICTIONARY).size assert position == len(stream) @@ -115,13 +149,13 @@ def test_the_ticks_reach_the_songs_length(self) -> None: tokens = ( LiteralToken(values=b"\x01\x02"), HoldToken(ticks=3), - PhraseToken(phrase_id=CHEAP_ID, ticks=4, transpose=0), + PhraseToken(phrase_id=CHEAP_ID, ticks=4, transpose=0, default=False), ) stream = emit(tokens) position = 0 covered = 0 for _ in tokens: - span = token_span(stream, position) + span = token_span(stream, position, DICTIONARY) covered += span.ticks position += span.size diff --git a/tests/unit/sampletones_player/driver/test_addresses.py b/tests/unit/sampletones_player/driver/test_addresses.py index 91d4b195d..cf364abf1 100644 --- a/tests/unit/sampletones_player/driver/test_addresses.py +++ b/tests/unit/sampletones_player/driver/test_addresses.py @@ -33,5 +33,9 @@ def test_the_song_follows_the_code(self) -> None: def test_a_longer_driver_moves_the_song_alone(self) -> None: shorter = DriverAddresses.for_code(CODE_LENGTH) longer = DriverAddresses.for_code(CODE_LENGTH * 2) - assert (longer.load, longer.init, longer.play) == (shorter.load, shorter.init, shorter.play) + assert (longer.load, longer.init, longer.play) == ( + shorter.load, + shorter.init, + shorter.play, + ) assert longer.song > shorter.song diff --git a/tests/unit/sampletones_player/driver/test_image.py b/tests/unit/sampletones_player/driver/test_image.py index a9151e1b0..13a04fb26 100644 --- a/tests/unit/sampletones_player/driver/test_image.py +++ b/tests/unit/sampletones_player/driver/test_image.py @@ -41,7 +41,10 @@ def test_the_song_begins_where_the_code_ends(self) -> None: def test_the_routines_answer_where_the_specification_states(self) -> None: image = DriverImage.load() - assert (image.addresses.init, image.addresses.play) == (INIT_ADDRESS, PLAY_ADDRESS) + assert (image.addresses.init, image.addresses.play) == ( + INIT_ADDRESS, + PLAY_ADDRESS, + ) def test_the_image_leads_with_a_jump_to_each_routine(self) -> None: assert DriverImage.load().code[: len(JUMP_TABLE) : 3] == bytes((JUMP_ABSOLUTE_OPCODE,) * 2) @@ -62,7 +65,9 @@ def test_an_init_routine_before_the_load_address_is_rejected(self) -> None: with pytest.raises(ValueError, match="the init routine lies at"): DriverImage(**image_fields(init=LOAD_ADDRESS - 1)) - def test_an_image_that_leads_with_anything_but_a_jump_is_rejected(self) -> None: + def test_an_image_that_leads_with_anything_but_a_jump_is_rejected( + self, + ) -> None: with pytest.raises(ValueError, match="rather than the jump"): DriverImage(**image_fields(code=bytes((RETURN_OPCODE,)) * len(CODE))) diff --git a/tests/unit/sampletones_player/export/test_backend.py b/tests/unit/sampletones_player/export/test_backend.py index 532a5e272..bf30a3fc1 100644 --- a/tests/unit/sampletones_player/export/test_backend.py +++ b/tests/unit/sampletones_player/export/test_backend.py @@ -32,7 +32,11 @@ STRING_FIELD_SIZE, TITLE_OFFSET, ) -from sampletones_player.specification.song import LOOP_TICK_OFFSET, NO_LOOP, TOTAL_TICKS_OFFSET +from sampletones_player.specification.song import ( + LOOP_TICK_OFFSET, + NO_LOOP, + TOTAL_TICKS_OFFSET, +) from sampletones_shared.exceptions import OperationCanceled, SongTooLargeError from sampletones_shared.paths.extensions import EXT_FILE_NSF from tests.suite.performance import ( @@ -153,7 +157,10 @@ def test_the_run_writes_the_destination_alone(self, backend: NSFBackend, tmp_pat destination = tmp_path / FILENAME request = player_sample( SAMPLE_NAME, - (lead_slice("lead", SOUNDING_TICKS), bass_slice("bass", SOUNDING_TICKS)), + ( + lead_slice("lead", SOUNDING_TICKS), + bass_slice("bass", SOUNDING_TICKS), + ), nes_frequency=NTSC_FREQUENCY, ) artifact = backend.write_sample(destination, request) @@ -161,13 +168,21 @@ def test_the_run_writes_the_destination_alone(self, backend: NSFBackend, tmp_pat def test_the_program_carries_its_driver(self, backend: NSFBackend, tmp_path: Path) -> None: destination = tmp_path / FILENAME - request = player_sample(SAMPLE_NAME, (lead_slice("lead", SOUNDING_TICKS),), nes_frequency=NTSC_FREQUENCY) + request = player_sample( + SAMPLE_NAME, + (lead_slice("lead", SOUNDING_TICKS),), + nes_frequency=NTSC_FREQUENCY, + ) backend.write_sample(destination, request) assert len(destination.read_bytes()) > HEADER_SIZE def test_the_reconstructions_name_lists_the_program(self, backend: NSFBackend, tmp_path: Path) -> None: destination = tmp_path / FILENAME - request = player_sample(SAMPLE_NAME, (lead_slice("lead", SOUNDING_TICKS),), nes_frequency=NTSC_FREQUENCY) + request = player_sample( + SAMPLE_NAME, + (lead_slice("lead", SOUNDING_TICKS),), + nes_frequency=NTSC_FREQUENCY, + ) backend.write_sample(destination, request) assert read_field(destination.read_bytes(), TITLE_OFFSET) == SAMPLE_NAME @@ -176,18 +191,30 @@ def test_an_export_is_credited_to_nobody(self, backend: NSFBackend, tmp_path: Pa listing the file leaves the line blank. """ destination = tmp_path / FILENAME - request = player_sample(SAMPLE_NAME, (lead_slice("lead", SOUNDING_TICKS),), nes_frequency=NTSC_FREQUENCY) + request = player_sample( + SAMPLE_NAME, + (lead_slice("lead", SOUNDING_TICKS),), + nes_frequency=NTSC_FREQUENCY, + ) backend.write_sample(destination, request) assert read_field(destination.read_bytes(), ARTIST_OFFSET) == "" def test_the_envelopes_cross_over_whole(self, backend: NSFBackend, tmp_path: Path) -> None: destination = tmp_path / FILENAME - request = player_sample(SAMPLE_NAME, (lead_slice("lead", SOUNDING_TICKS),), nes_frequency=NTSC_FREQUENCY) + request = player_sample( + SAMPLE_NAME, + (lead_slice("lead", SOUNDING_TICKS),), + nes_frequency=NTSC_FREQUENCY, + ) assert backend.write_sample(destination, request).truncation is None def test_a_destination_reaches_a_directory_the_run_creates(self, backend: NSFBackend, tmp_path: Path) -> None: destination = tmp_path / "exports" / FILENAME - request = player_sample(SAMPLE_NAME, (lead_slice("lead", SOUNDING_TICKS),), nes_frequency=NTSC_FREQUENCY) + request = player_sample( + SAMPLE_NAME, + (lead_slice("lead", SOUNDING_TICKS),), + nes_frequency=NTSC_FREQUENCY, + ) backend.write_sample(destination, request) assert destination.is_file() @@ -218,7 +245,10 @@ def test_a_slice_plays_the_program_its_reconstruction_would(self, backend: NSFBa together = tmp_path / "together.nsf" backend.write_instrument(alone, instrument) - backend.write_sample(together, player_sample(SAMPLE_NAME, (instrument,), nes_frequency=NTSC_FREQUENCY)) + backend.write_sample( + together, + player_sample(SAMPLE_NAME, (instrument,), nes_frequency=NTSC_FREQUENCY), + ) assert alone.read_bytes() == together.read_bytes() @@ -355,7 +385,11 @@ def test_the_run_names_each_stage_in_the_order_it_reaches_it( tmp_path: Path, ) -> None: reporter: RecordingReporter[ExportProgress] = RecordingReporter() - request = player_sample(SAMPLE_NAME, (lead_slice("lead", SOUNDING_TICKS),), nes_frequency=NTSC_FREQUENCY) + request = player_sample( + SAMPLE_NAME, + (lead_slice("lead", SOUNDING_TICKS),), + nes_frequency=NTSC_FREQUENCY, + ) backend.write_sample(tmp_path / FILENAME, request, reporter) assert reported_stages(reporter.reports) == [ ExportStage.WALKING, @@ -370,7 +404,11 @@ def test_the_compression_reports_the_bytes_it_has_laid_down( ) -> None: """A search ends where the song runs out of phrases that pay, so it counts bytes alone.""" reporter: RecordingReporter[ExportProgress] = RecordingReporter() - request = player_sample(SAMPLE_NAME, (lead_slice("lead", SOUNDING_TICKS),), nes_frequency=NTSC_FREQUENCY) + request = player_sample( + SAMPLE_NAME, + (lead_slice("lead", SOUNDING_TICKS),), + nes_frequency=NTSC_FREQUENCY, + ) backend.write_sample(tmp_path / FILENAME, request, reporter) compressing = [report for report in reporter.reports if report.stage == ExportStage.COMPRESSING] assert compressing and all(report.total is None for report in compressing) @@ -391,7 +429,11 @@ class TestWithdrawingARun: def test_a_withdrawn_run_writes_nothing(self, backend: NSFBackend, tmp_path: Path) -> None: destination = tmp_path / FILENAME reporter: RecordingReporter[ExportProgress] = RecordingReporter(withdraw_at=WITHDRAWN_WHILE_WALKING) - request = player_sample(SAMPLE_NAME, (lead_slice("lead", SOUNDING_TICKS),), nes_frequency=NTSC_FREQUENCY) + request = player_sample( + SAMPLE_NAME, + (lead_slice("lead", SOUNDING_TICKS),), + nes_frequency=NTSC_FREQUENCY, + ) with pytest.raises(OperationCanceled): backend.write_sample(destination, request, reporter) @@ -404,7 +446,11 @@ def test_a_run_withdrawn_mid_compression_writes_nothing( ) -> None: destination = tmp_path / FILENAME reporter: RecordingReporter[ExportProgress] = RecordingReporter(withdraw_at=WITHDRAWN_WHILE_COMPRESSING) - request = player_sample(SAMPLE_NAME, (lead_slice("lead", SOUNDING_TICKS),), nes_frequency=NTSC_FREQUENCY) + request = player_sample( + SAMPLE_NAME, + (lead_slice("lead", SOUNDING_TICKS),), + nes_frequency=NTSC_FREQUENCY, + ) with pytest.raises(OperationCanceled): backend.write_sample(destination, request, reporter) @@ -434,7 +480,10 @@ def repeated_sample() -> SampleExport: """A reconstruction holding one note, which every scheme past the first writes as a run.""" return player_sample( SAMPLE_NAME, - (lead_slice("lead", REPEATED_TICKS), bass_slice("bass", REPEATED_TICKS)), + ( + lead_slice("lead", REPEATED_TICKS), + bass_slice("bass", REPEATED_TICKS), + ), nes_frequency=NTSC_FREQUENCY, ) @@ -459,27 +508,42 @@ def test_the_file_is_listed_under_the_chosen_text( expected: str, ) -> None: destination = tmp_path / FILENAME - program = chosen_program(channels=ALL_CHANNELS, loop_tick=SONG_START, scheme=CompressionScheme.SEARCH) + program = chosen_program( + channels=ALL_CHANNELS, + loop_tick=SONG_START, + scheme=CompressionScheme.SEARCH, + ) backend.choosing(program).write_project(destination, ProjectExport(project=drum_project())) assert read_field(destination.read_bytes(), offset) == expected def test_a_song_chosen_to_play_once_stops_at_its_end(self, backend: NSFBackend, tmp_path: Path) -> None: destination = tmp_path / FILENAME - program = chosen_program(channels=ALL_CHANNELS, loop_tick=None, scheme=CompressionScheme.SEARCH) + program = chosen_program( + channels=ALL_CHANNELS, + loop_tick=None, + scheme=CompressionScheme.SEARCH, + ) backend.choosing(program).write_project(destination, ProjectExport(project=drum_project())) assert written_loop_tick(destination.read_bytes()) == NO_LOOP def test_a_reconstruction_repeats_where_the_program_says(self, backend: NSFBackend, tmp_path: Path) -> None: """The slices play once, and the program asks for them to come round all the same.""" destination = tmp_path / FILENAME - program = chosen_program(channels=ALL_CHANNELS, loop_tick=SONG_START, scheme=CompressionScheme.SEARCH) + program = chosen_program( + channels=ALL_CHANNELS, + loop_tick=SONG_START, + scheme=CompressionScheme.SEARCH, + ) backend.choosing(program).write_sample(destination, repeated_sample()) assert written_loop_tick(destination.read_bytes()) == SONG_START def test_a_lighter_scheme_writes_a_larger_file(self, backend: NSFBackend, tmp_path: Path) -> None: spelled_out = tmp_path / "none.nsf" runs = tmp_path / "runs.nsf" - for destination, scheme in ((spelled_out, CompressionScheme.NONE), (runs, CompressionScheme.RUNS)): + for destination, scheme in ( + (spelled_out, CompressionScheme.NONE), + (runs, CompressionScheme.RUNS), + ): program = chosen_program(channels=ALL_CHANNELS, loop_tick=None, scheme=scheme) backend.choosing(program).write_sample(destination, repeated_sample()) @@ -491,6 +555,12 @@ def test_choosing_a_program_leaves_the_backend_it_came_from_as_it_was( tmp_path: Path, ) -> None: destination = tmp_path / FILENAME - backend.choosing(chosen_program(channels=ALL_CHANNELS, loop_tick=None, scheme=CompressionScheme.NONE)) + backend.choosing( + chosen_program( + channels=ALL_CHANNELS, + loop_tick=None, + scheme=CompressionScheme.NONE, + ) + ) backend.write_project(destination, ProjectExport(project=drum_project())) assert read_field(destination.read_bytes(), TITLE_OFFSET) == PROJECT_TITLE diff --git a/tests/unit/sampletones_player/export/test_program.py b/tests/unit/sampletones_player/export/test_program.py index c40dc3bc2..3eb0cd7cf 100644 --- a/tests/unit/sampletones_player/export/test_program.py +++ b/tests/unit/sampletones_player/export/test_program.py @@ -8,7 +8,11 @@ from sampletones_core.project.project import Project from sampletones_player.builder import SONG_START, loop_tick_from_instruments from sampletones_player.compression.scheme import CompressionScheme -from sampletones_player.export.program import DEFAULT_SCHEME, NO_ARTIST, NSFProgram +from sampletones_player.export.program import ( + DEFAULT_SCHEME, + NO_ARTIST, + NSFProgram, +) from sampletones_player.nsf.information import NSFInformation from sampletones_shared.application import SAMPLETONES_COPYRIGHT from tests.suite.player import ( @@ -67,9 +71,14 @@ def test_it_is_compressed_under_the_default_scheme(self, project: Project) -> No class TestTheProgramASampleStates: """What a reconstruction's slices are written as when nobody chose otherwise.""" - def test_it_is_listed_under_the_reconstructions_name_credited_to_nobody(self) -> None: + def test_it_is_listed_under_the_reconstructions_name_credited_to_nobody( + self, + ) -> None: information = NSFProgram.for_sample(sample(loop=False)).information - assert (information.title, information.artist) == (SAMPLE_NAME, NO_ARTIST) + assert (information.title, information.artist) == ( + SAMPLE_NAME, + NO_ARTIST, + ) def test_it_sounds_every_channel(self) -> None: assert NSFProgram.for_sample(sample(loop=False)).channels == ALL_CHANNELS diff --git a/tests/unit/sampletones_player/nsf/test_header.py b/tests/unit/sampletones_player/nsf/test_header.py index 2a4ec7097..cb4139495 100644 --- a/tests/unit/sampletones_player/nsf/test_header.py +++ b/tests/unit/sampletones_player/nsf/test_header.py @@ -150,14 +150,19 @@ def test_the_field_carries_its_text(self, test_case: TestCase) -> None: field = read_string(header(), test_case.offset) assert field == test_case.expected.encode("utf-8").ljust(STRING_FIELD_SIZE, b"\x00") - def test_text_longer_than_the_field_is_written_as_much_as_fits_before_its_terminator(self) -> None: + def test_text_longer_than_the_field_is_written_as_much_as_fits_before_its_terminator( + self, + ) -> None: """A player reads each field up to its NUL, so the field keeps a byte for one.""" overlong = "A" * (STRING_FIELD_SIZE * 2) data = header_to_bytes(NSFInformation(title=overlong, artist=ARTIST), ADDRESSES) assert read_string(data, TITLE_OFFSET) == overlong.encode("utf-8")[:STRING_TEXT_SIZE] + b"\x00" def test_the_fields_stand_back_to_back(self) -> None: - assert (ARTIST_OFFSET - TITLE_OFFSET, COPYRIGHT_OFFSET - ARTIST_OFFSET) == ( + assert ( + ARTIST_OFFSET - TITLE_OFFSET, + COPYRIGHT_OFFSET - ARTIST_OFFSET, + ) == ( STRING_FIELD_SIZE, STRING_FIELD_SIZE, ) @@ -169,7 +174,9 @@ class TestHeaderPlayback: def test_the_ntsc_period_is_the_one_the_specification_names(self) -> None: assert read_word(header(), NTSC_PERIOD_OFFSET) == NTSC_PLAY_PERIOD_MICROSECONDS - def test_the_ntsc_period_asks_for_the_rate_the_schedule_counts_in(self) -> None: + def test_the_ntsc_period_asks_for_the_rate_the_schedule_counts_in( + self, + ) -> None: """A player reading the field and one driving from the frame run a stream at one speed.""" requested = Fraction(MICROSECONDS_PER_SECOND, read_word(header(), NTSC_PERIOD_OFFSET)) assert abs(requested - NTSC_FRAME_RATE) / NTSC_FRAME_RATE < RATE_TOLERANCE diff --git a/tests/unit/sampletones_player/nsf/test_information.py b/tests/unit/sampletones_player/nsf/test_information.py index a8a9cfed7..8d482da4f 100644 --- a/tests/unit/sampletones_player/nsf/test_information.py +++ b/tests/unit/sampletones_player/nsf/test_information.py @@ -3,8 +3,16 @@ import pytest -from sampletones_player.nsf.information import TEXT_ENCODING, NSFInformation, field_size, fit_field -from sampletones_player.specification.nsf import STRING_FIELD_SIZE, STRING_TEXT_SIZE +from sampletones_player.nsf.information import ( + TEXT_ENCODING, + NSFInformation, + field_size, + fit_field, +) +from sampletones_player.specification.nsf import ( + STRING_FIELD_SIZE, + STRING_TEXT_SIZE, +) from tests.suite.base import BaseTestSuite from tests.suite.case import BaseRegularTestCase @@ -23,8 +31,16 @@ class TestCase(BaseRegularTestCase): test_cases = ( TestCase(label="short text", text=SHORT_TEXT, expected=SHORT_TEXT), TestCase(label="empty text", text="", expected=""), - TestCase(label="exactly the room", text="A" * STRING_TEXT_SIZE, expected="A" * STRING_TEXT_SIZE), - TestCase(label="one byte over", text="A" * STRING_FIELD_SIZE, expected="A" * STRING_TEXT_SIZE), + TestCase( + label="exactly the room", + text="A" * STRING_TEXT_SIZE, + expected="A" * STRING_TEXT_SIZE, + ), + TestCase( + label="one byte over", + text="A" * STRING_FIELD_SIZE, + expected="A" * STRING_TEXT_SIZE, + ), TestCase( label="a character straddling the edge", text="A" * (STRING_TEXT_SIZE - 1) + WIDE_CHARACTER, @@ -55,4 +71,8 @@ class TestNSFInformation: def test_every_field_is_held_as_it_fits(self) -> None: overlong = "B" * STRING_FIELD_SIZE information = NSFInformation(title=overlong, artist=overlong, copyright=overlong) - assert {information.title, information.artist, information.copyright} == {fit_field(overlong)} + assert { + information.title, + information.artist, + information.copyright, + } == {fit_field(overlong)} diff --git a/tests/unit/sampletones_player/nsf/test_layout.py b/tests/unit/sampletones_player/nsf/test_layout.py index bc0f010cb..fe6dbc1db 100644 --- a/tests/unit/sampletones_player/nsf/test_layout.py +++ b/tests/unit/sampletones_player/nsf/test_layout.py @@ -4,12 +4,16 @@ from sampletones_player.nsf.layout import SongLayout from sampletones_player.song import Song from sampletones_player.specification.compression import ( + PHRASE_DEFAULT_SIZE, PHRASE_LENGTH_SIZE, PHRASE_TABLE_COUNT_SIZE, PHRASE_TABLE_ENTRY_SIZE, - PLANE_COUNT, ) -from sampletones_player.specification.song import ABSENT_STREAM, SONG_HEADER_SIZE +from sampletones_player.specification.planes import PLANE_COUNT +from sampletones_player.specification.song import ( + ABSENT_STREAM, + SONG_HEADER_SIZE, +) from tests.suite.player import ( PLAYER_FULL_VOLUME, PLAYER_OCTAVE_UP_TIMER, @@ -32,7 +36,11 @@ def figure_song(loop_tick: int) -> Song: - return player_song(resting_streams(FIGURE * FIGURE_REPEATS), NTSC_FREQUENCY, loop_tick=loop_tick) + return player_song( + resting_streams(FIGURE * FIGURE_REPEATS), + NTSC_FREQUENCY, + loop_tick=loop_tick, + ) def present(song: Song, layout: SongLayout) -> List[Tuple[bytes, int]]: @@ -67,7 +75,7 @@ def test_the_first_stream_follows_the_last_body(self) -> None: song = figure_song(LOOP_TICK) layout = SongLayout.of(song) last = song.planes.phrases[len(song.planes.phrases) - 1] - assert layout.streams[0] == layout.bodies[-1] + PHRASE_LENGTH_SIZE + last.length + assert layout.streams[0] == layout.bodies[-1] + PHRASE_LENGTH_SIZE + PHRASE_DEFAULT_SIZE + last.length def test_each_stream_follows_the_one_before_it(self) -> None: song = figure_song(LOOP_TICK) @@ -81,7 +89,9 @@ def test_the_block_ends_behind_the_last_stream(self) -> None: stream, offset = present(song, layout)[-1] assert layout.size == offset + len(stream) - def test_an_absent_plane_states_the_sentinel_for_its_stream_and_its_entry(self) -> None: + def test_an_absent_plane_states_the_sentinel_for_its_stream_and_its_entry( + self, + ) -> None: song = figure_song(LOOP_TICK) layout = SongLayout.of(song) absent = [plane for plane, stream in enumerate(song.planes.streams) if not stream] @@ -99,13 +109,19 @@ def test_every_plane_states_an_entry(self) -> None: def test_an_entry_stands_at_the_token_the_loop_tick_starts(self) -> None: song = figure_song(LOOP_TICK) layout = SongLayout.of(song) - entered = song.planes.entries(decode_planes(song.planes).positions(LOOP_TICK)) + entered = song.planes.loop_entries assert layout.loop_entries == tuple( ABSENT_STREAM if entry is None else offset + entry for offset, entry in zip(layout.streams, entered) ) - def test_a_song_that_stops_re_enters_at_each_streams_own_start(self) -> None: - song = player_song(resting_streams(FIGURE * FIGURE_REPEATS), NTSC_FREQUENCY, loop_tick=None) + def test_a_song_that_stops_re_enters_at_each_streams_own_start( + self, + ) -> None: + song = player_song( + resting_streams(FIGURE * FIGURE_REPEATS), + NTSC_FREQUENCY, + loop_tick=None, + ) layout = SongLayout.of(song) assert layout.loop_entries == layout.streams diff --git a/tests/unit/sampletones_player/nsf/test_song.py b/tests/unit/sampletones_player/nsf/test_song.py index 17d03d46e..ebabedc21 100644 --- a/tests/unit/sampletones_player/nsf/test_song.py +++ b/tests/unit/sampletones_player/nsf/test_song.py @@ -7,10 +7,14 @@ from sampletones_player.compression.decode import decode_planes from sampletones_player.compression.planes.order import PlaneOrder from sampletones_player.nsf.layout import NAME_SEPARATOR, SongLayout -from sampletones_player.nsf.song import song_to_bytes +from sampletones_player.nsf.song import STATED_BEYOND_THE_FIRST, song_to_bytes from sampletones_player.song import Song from sampletones_player.specification.binary import WORD_SIZE -from sampletones_player.specification.compression import PLANE_COUNT +from sampletones_player.specification.compression import ( + PHRASE_DEFAULT_SIZE, + PHRASE_LENGTH_SIZE, +) +from sampletones_player.specification.planes import PLANE_COUNT from sampletones_player.specification.song import ( ABSENT_STREAM, LOOP_ENTRIES_OFFSET, @@ -87,9 +91,12 @@ class TestSongBytes: The layout is the contract the driver reads the song through, so the literal states it in full: the header, the timer every pitch sounds at, the dictionary the tokens name, and one - token stream per plane the block holds. The song bends nowhere, so each bend plane is absent - and its header entries carry the sentinel. The timer table is named rather than transcribed, - since it is the tuning's own table and the block carries whatever that table holds. + token stream per plane the block holds. The song bends nowhere, so each bend plane is absent; + its second pulse channel and its noise channel rest throughout, so both control planes stand + at the value their register fixes; and its triangle never sounds, so its value plane names the + index that stands for silence. Each of those is absent, and its header entries carry the + sentinel. The timer table is named rather than transcribed, since it is the tuning's own table + and the block carries whatever that table holds. """ EXPECTED_HEADER: Final[bytes] = ( @@ -97,36 +104,28 @@ class TestSongBytes: b"\xca\x7f" b"\x02\x00" b"\xff\xff" - b"\x37\x00" - b"\x07\x01" - b"\x08\x01\x0b\x01\xff\xff" - b"\x0e\x01\x11\x01\xff\xff" - b"\x14\x01\x17\x01\xff\xff" - b"\x1a\x01\x1d\x01" - b"\x08\x01\x0b\x01\xff\xff" - b"\x0e\x01\x11\x01\xff\xff" - b"\x14\x01\x17\x01\xff\xff" - b"\x1a\x01\x1d\x01" + b"\x33\x00" + b"\x03\x01" + b"\x04\x01\x07\x01\xff\xff" + b"\xff\xff\x0a\x01\xff\xff" + b"\xff\xff\xff\xff" + b"\xff\xff\x0d\x01" + b"\x04\x01\x07\x01\xff\xff" + b"\xff\xff\x0a\x01\xff\xff" + b"\xff\xff\xff\xff" + b"\xff\xff\x0d\x01" ) - EXPECTED_STREAMS: Final[bytes] = ( - b"\x00" - b"\x41\x3f\x30" - b"\x40\x21\x00" - b"\x40\x30\x00" - b"\x40\x21\x00" - b"\x40\x80\x00" - b"\x40\x21\x00" - b"\x40\x30\x00" - b"\x40\x0a\x00" - ) + EXPECTED_STREAMS: Final[bytes] = b"\x00" b"\x41\x0f\x00" b"\x40\x21\x00" b"\x40\x21\x00" b"\x40\x1a" def test_the_song_serializes_to_the_expected_bytes(self) -> None: song = two_tick_song(HALF_RATE_FREQUENCY) expected = self.EXPECTED_HEADER + song.pitches.data + self.EXPECTED_STREAMS assert song_to_bytes(song, PROGRAM_AREA_BYTES) == expected - def test_the_header_runs_to_the_length_the_offsets_are_read_at(self) -> None: + def test_the_header_runs_to_the_length_the_offsets_are_read_at( + self, + ) -> None: assert len(self.EXPECTED_HEADER) == SONG_HEADER_SIZE @@ -160,7 +159,11 @@ def test_the_step_reaches_the_header_as_the_driver_holds_it(self, test_case: Tes assert read_word(data, STEP_FRACTION_OFFSET) == step.fraction def test_the_header_states_the_songs_length(self) -> None: - song = player_song(resting_streams((SOUNDING, OCTAVE_UP, RESTING)), NTSC_FREQUENCY, loop_tick=None) + song = player_song( + resting_streams((SOUNDING, OCTAVE_UP, RESTING)), + NTSC_FREQUENCY, + loop_tick=None, + ) data = song_to_bytes(song, PROGRAM_AREA_BYTES) assert read_word(data, TOTAL_TICKS_OFFSET) == song.ticks @@ -200,8 +203,10 @@ def test_each_entry_reaches_that_phrases_length_and_body(self) -> None: data = song_to_bytes(song, PROGRAM_AREA_BYTES) layout = SongLayout.of(song) for phrase, offset in zip(song.planes.phrases.phrases, layout.bodies): + body = offset + PHRASE_LENGTH_SIZE + PHRASE_DEFAULT_SIZE assert data[offset] == phrase.length - assert data[offset + 1 : offset + 1 + phrase.length] == phrase.body + assert data[offset + PHRASE_LENGTH_SIZE] == phrase.default - STATED_BEYOND_THE_FIRST + assert data[body : body + phrase.length] == phrase.body def test_the_entries_stand_where_the_table_states(self) -> None: song = spelled_phrase_song() @@ -253,10 +258,12 @@ def test_the_streams_fill_the_song_to_its_last_byte(self) -> None: class TestLoopEntries: """Where each plane's stream is re-entered once the song repeats.""" - def test_a_song_that_repeats_states_the_token_its_loop_tick_starts(self) -> None: + def test_a_song_that_repeats_states_the_token_its_loop_tick_starts( + self, + ) -> None: song = repeating_song() data = song_to_bytes(song, PROGRAM_AREA_BYTES) - entered = song.planes.entries(decode_planes(song.planes).positions(LOOP_TICK)) + entered = song.planes.loop_entries assert loop_entries(data) == tuple( ABSENT_STREAM if entry is None else offset + entry for offset, entry in zip(stream_offsets(data), entered) ) @@ -282,12 +289,16 @@ def test_a_song_past_the_available_space_raises(self) -> None: with pytest.raises(SongTooLargeError): song_to_bytes(song, len(data) - 1) - def test_a_song_filling_the_available_space_exactly_is_written(self) -> None: + def test_a_song_filling_the_available_space_exactly_is_written( + self, + ) -> None: song = two_tick_song(NTSC_FREQUENCY) data = song_to_bytes(song, PROGRAM_AREA_BYTES) assert song_to_bytes(song, len(data)) == data - def test_a_song_reaching_past_the_offset_field_names_what_overflowed(self) -> None: + def test_a_song_reaching_past_the_offset_field_names_what_overflowed( + self, + ) -> None: """A block given exactly the room it takes is still refused where an offset overflows.""" song = spelled_song(MAX_BLOCK_OFFSET, NTSC_FREQUENCY) with pytest.raises(SongTooLargeError) as overflow: diff --git a/tests/unit/sampletones_player/registers/test_channel.py b/tests/unit/sampletones_player/registers/test_channel.py index 7e3222e02..7d9518e35 100644 --- a/tests/unit/sampletones_player/registers/test_channel.py +++ b/tests/unit/sampletones_player/registers/test_channel.py @@ -9,7 +9,10 @@ PulseInstruction, TriangleInstruction, ) -from sampletones_player.registers.channel import channel_instructions, channel_registers +from sampletones_player.registers.channel import ( + channel_instructions, + channel_registers, +) from sampletones_player.specification.registers import ( TRIANGLE_COUNTER_CONTROL, TRIANGLE_SOUNDING_RELOAD, @@ -59,15 +62,25 @@ def test_a_stream_of_another_channels_type_raises(self) -> None: class TestChannelRegisters: """Naming the channel is the whole of what it takes to encode a stream.""" - def test_a_sounding_channel_carries_a_tick_per_instruction_and_a_release(self) -> None: - registers = channel_registers(ChannelName.PULSE1, {ChannelName.PULSE1: melody()}, PLAYER_TIMER_TABLE) + def test_a_sounding_channel_carries_a_tick_per_instruction_and_a_release( + self, + ) -> None: + registers = channel_registers( + ChannelName.PULSE1, + {ChannelName.PULSE1: melody()}, + PLAYER_TIMER_TABLE, + ) assert len(registers) == SOUNDING_TICKS + 1 def test_a_channel_the_song_leaves_out_rests_for_a_tick(self) -> None: assert len(channel_registers(ChannelName.PULSE2, {}, PLAYER_TIMER_TABLE)) == 1 def test_a_pitch_reaches_the_timer_the_table_states(self) -> None: - registers = channel_registers(ChannelName.PULSE1, {ChannelName.PULSE1: melody()}, PLAYER_TIMER_TABLE) + registers = channel_registers( + ChannelName.PULSE1, + {ChannelName.PULSE1: melody()}, + PLAYER_TIMER_TABLE, + ) timer = PLAYER_TIMER_TABLE[PLAYER_REFERENCE_PITCH] assert registers[0].values[1:] == (timer & 0xFF, timer >> 8) @@ -91,6 +104,12 @@ def test_the_noise_channel_answers_in_its_own_registers(self) -> None: ) assert registers[0].control & 0x0F == NOISE_VOLUME - def test_a_channel_holding_another_channels_instructions_raises(self) -> None: + def test_a_channel_holding_another_channels_instructions_raises( + self, + ) -> None: with pytest.raises(TypeError): - channel_registers(ChannelName.TRIANGLE, {ChannelName.TRIANGLE: melody()}, PLAYER_TIMER_TABLE) + channel_registers( + ChannelName.TRIANGLE, + {ChannelName.TRIANGLE: melody()}, + PLAYER_TIMER_TABLE, + ) diff --git a/tests/unit/sampletones_player/registers/test_dividers.py b/tests/unit/sampletones_player/registers/test_dividers.py index aa6f31d8b..65b15fbf9 100644 --- a/tests/unit/sampletones_player/registers/test_dividers.py +++ b/tests/unit/sampletones_player/registers/test_dividers.py @@ -1,8 +1,16 @@ import pytest -from sampletones_core.constants.general import MAX_PITCH, MAX_TIMER, MIN_PITCH, MIN_TIMER +from sampletones_core.constants.general import ( + MAX_PITCH, + MAX_TIMER, + MIN_PITCH, + MIN_TIMER, +) from sampletones_core.timers.nearest import nearest_pitch -from sampletones_player.registers.dividers import anchored_pitches, bent_dividers +from sampletones_player.registers.dividers import ( + anchored_pitches, + bent_dividers, +) from sampletones_player.specification.binary import SIGNED_BYTE_LIMIT from tests.suite.player import PLAYER_REFERENCE_PITCH, PLAYER_TIMER_TABLE @@ -17,14 +25,20 @@ def test_an_unbent_frame_sounds_its_note_s_own_divider(self) -> None: def test_every_frame_moves_by_its_own_bend(self) -> None: offsets = (3, -3, 40) - dividers = bent_dividers((PLAYER_REFERENCE_PITCH,) * len(offsets), offsets, PLAYER_TIMER_TABLE) + dividers = bent_dividers( + (PLAYER_REFERENCE_PITCH,) * len(offsets), + offsets, + PLAYER_TIMER_TABLE, + ) assert dividers == tuple(PLAYER_TIMER_TABLE[PLAYER_REFERENCE_PITCH] + offset for offset in offsets) def test_a_bend_past_the_register_stops_at_its_edge(self) -> None: dividers = bent_dividers((MIN_PITCH, MAX_PITCH), (MAX_TIMER, -MAX_TIMER), PLAYER_TIMER_TABLE) assert dividers == (MAX_TIMER, MIN_TIMER) - def test_notes_and_bends_covering_different_frames_are_refused(self) -> None: + def test_notes_and_bends_covering_different_frames_are_refused( + self, + ) -> None: with pytest.raises(ValueError): bent_dividers((PLAYER_REFERENCE_PITCH,), (0, 0), PLAYER_TIMER_TABLE) @@ -32,17 +46,23 @@ def test_notes_and_bends_covering_different_frames_are_refused(self) -> None: class TestAnchoredPitches: """A divider is counted from the frame's own note wherever the bend fits the plane's byte.""" - def test_a_bend_past_halfway_is_counted_from_the_frames_own_note(self) -> None: + def test_a_bend_past_halfway_is_counted_from_the_frames_own_note( + self, + ) -> None: divider = PLAYER_TIMER_TABLE[PLAYER_REFERENCE_PITCH] - SIGNED_BYTE_LIMIT + 1 assert nearest_pitch(PLAYER_TIMER_TABLE, divider).pitch != PLAYER_REFERENCE_PITCH assert anchored_pitches((PLAYER_REFERENCE_PITCH,), (divider,), PLAYER_TIMER_TABLE) == (PLAYER_REFERENCE_PITCH,) - def test_a_bend_past_the_byte_is_counted_from_the_nearest_pitch(self) -> None: + def test_a_bend_past_the_byte_is_counted_from_the_nearest_pitch( + self, + ) -> None: divider = PLAYER_TIMER_TABLE[PLAYER_REFERENCE_PITCH] - SIGNED_BYTE_LIMIT - 1 assert anchored_pitches((PLAYER_REFERENCE_PITCH,), (divider,), PLAYER_TIMER_TABLE) == ( nearest_pitch(PLAYER_TIMER_TABLE, divider).pitch, ) - def test_notes_and_dividers_covering_different_frames_are_refused(self) -> None: + def test_notes_and_dividers_covering_different_frames_are_refused( + self, + ) -> None: with pytest.raises(ValueError): anchored_pitches((PLAYER_REFERENCE_PITCH,), (), PLAYER_TIMER_TABLE) diff --git a/tests/unit/sampletones_player/registers/test_pulse.py b/tests/unit/sampletones_player/registers/test_pulse.py index 07550a990..4152a5aa9 100644 --- a/tests/unit/sampletones_player/registers/test_pulse.py +++ b/tests/unit/sampletones_player/registers/test_pulse.py @@ -18,7 +18,10 @@ from sampletones_core.instructions import PulseInstruction from sampletones_core.timers.nearest import nearest_pitch from sampletones_player.registers.pulse import PulseRegisters -from sampletones_player.specification.registers import MAX_REGISTER_VALUE, TIMER_HIGH_SHIFT +from sampletones_player.specification.registers import ( + MAX_REGISTER_VALUE, + TIMER_HIGH_SHIFT, +) from tests.suite.base import BaseTestSuite from tests.suite.case import BaseAutolabelTestCase from tests.suite.player import ( @@ -66,7 +69,11 @@ def test_a_tick_states_control_then_timer(self) -> None: instructions = [sounding_pulse(PLAYER_REFERENCE_PITCH, MAX_VOLUME, MAX_DUTY_CYCLE)] registers = PulseRegisters.from_instructions(instructions, PLAYER_TIMER_TABLE)[0] timer = PLAYER_TIMER_TABLE[PLAYER_REFERENCE_PITCH] - assert registers.values == (0xFF, timer & MAX_REGISTER_VALUE, timer >> TIMER_HIGH_SHIFT) + assert registers.values == ( + 0xFF, + timer & MAX_REGISTER_VALUE, + timer >> TIMER_HIGH_SHIFT, + ) class TestPulseControlByte(BaseTestSuite): @@ -132,7 +139,10 @@ def test_rest_keeps_the_duty_cycle(self) -> None: def test_rest_keeps_the_timer(self) -> None: sounding, resting = self.encode_note_then_rest()[:2] - assert (resting.timer_low, resting.timer_high) == (sounding.timer_low, sounding.timer_high) + assert (resting.timer_low, resting.timer_high) == ( + sounding.timer_low, + sounding.timer_high, + ) class TestPulseReleaseTick: @@ -145,7 +155,10 @@ def test_pulse_gains_a_silent_closing_tick(self) -> None: assert registers[-1].control & 0x0F == 0 def test_a_sample_ending_in_a_rest_gains_no_extra_tick(self) -> None: - instructions = [sounding_pulse(PLAYER_REFERENCE_PITCH, MAX_VOLUME, 0), silent_pulse()] + instructions = [ + sounding_pulse(PLAYER_REFERENCE_PITCH, MAX_VOLUME, 0), + silent_pulse(), + ] assert len(PulseRegisters.from_instructions(instructions, PLAYER_TIMER_TABLE)) == 2 def test_no_instructions_encode_to_no_ticks(self) -> None: @@ -192,8 +205,13 @@ def test_the_timer_is_the_divider_the_generator_sounds( expected >> TIMER_HIGH_SHIFT, ) - def test_a_bend_within_a_byte_is_counted_from_the_frames_own_note(self) -> None: - instructions = [bent_pulse(PLAYER_REFERENCE_PITCH, -40, 0), bent_pulse(PLAYER_REFERENCE_PITCH, 0, 7)] + def test_a_bend_within_a_byte_is_counted_from_the_frames_own_note( + self, + ) -> None: + instructions = [ + bent_pulse(PLAYER_REFERENCE_PITCH, -40, 0), + bent_pulse(PLAYER_REFERENCE_PITCH, 0, 7), + ] registers = PulseRegisters.from_instructions(instructions, PLAYER_TIMER_TABLE) assert [tick.anchor for tick in registers] == [PLAYER_REFERENCE_PITCH] * len(registers) @@ -204,11 +222,23 @@ def test_a_bend_past_a_byte_is_counted_from_the_nearest_pitch(self) -> None: assert tick.anchor != PLAYER_REFERENCE_PITCH def test_a_rest_holds_the_bent_divider(self) -> None: - instructions = [bent_pulse(PLAYER_REFERENCE_PITCH, 9, 1), silent_pulse()] + instructions = [ + bent_pulse(PLAYER_REFERENCE_PITCH, 9, 1), + silent_pulse(), + ] sounding, resting = PulseRegisters.from_instructions(instructions, PLAYER_TIMER_TABLE)[:2] - assert (resting.timer_low, resting.timer_high) == (sounding.timer_low, sounding.timer_high) + assert (resting.timer_low, resting.timer_high) == ( + sounding.timer_low, + sounding.timer_high, + ) def test_a_rest_before_the_first_note_takes_its_bent_divider(self) -> None: - instructions = [silent_pulse(), bent_pulse(PLAYER_REFERENCE_PITCH, -9, 0)] + instructions = [ + silent_pulse(), + bent_pulse(PLAYER_REFERENCE_PITCH, -9, 0), + ] resting, sounding = PulseRegisters.from_instructions(instructions, PLAYER_TIMER_TABLE)[:2] - assert (resting.timer_low, resting.timer_high) == (sounding.timer_low, sounding.timer_high) + assert (resting.timer_low, resting.timer_high) == ( + sounding.timer_low, + sounding.timer_high, + ) diff --git a/tests/unit/sampletones_player/registers/test_streams.py b/tests/unit/sampletones_player/registers/test_streams.py index 69a1cb7de..7e0256afb 100644 --- a/tests/unit/sampletones_player/registers/test_streams.py +++ b/tests/unit/sampletones_player/registers/test_streams.py @@ -31,7 +31,11 @@ def test_the_longest_channel_states_the_length(self) -> None: def test_a_channel_past_its_end_holds_its_final_values(self) -> None: streams = resting_streams((SOUNDING, RESTING)) _, pulse2, triangle, noise = streams.at(1) - assert (pulse2, triangle, noise) == (streams.pulse2[0], streams.triangle[0], streams.noise[0]) + assert (pulse2, triangle, noise) == ( + streams.pulse2[0], + streams.triangle[0], + streams.noise[0], + ) def test_every_channel_reads_its_own_tick_while_it_lasts(self) -> None: streams = resting_streams((SOUNDING, OCTAVE_UP)) @@ -46,13 +50,20 @@ def test_padding_leaves_a_full_length_channel_as_it_stands(self) -> None: streams = resting_streams((SOUNDING, OCTAVE_UP, RESTING)) assert streams.padded[0] == (SOUNDING, OCTAVE_UP, RESTING) - def test_padding_repeats_the_final_values_of_a_shorter_channel(self) -> None: + def test_padding_repeats_the_final_values_of_a_shorter_channel( + self, + ) -> None: streams = resting_streams((SOUNDING, OCTAVE_UP, RESTING)) assert streams.padded[3] == (streams.noise[0],) * 3 def test_the_streams_stand_in_channel_order(self) -> None: streams = resting_streams((SOUNDING,)) - assert streams.ordered == (streams.pulse1, streams.pulse2, streams.triangle, streams.noise) + assert streams.ordered == ( + streams.pulse1, + streams.pulse2, + streams.triangle, + streams.noise, + ) def test_a_channel_without_a_tick_raises(self) -> None: with pytest.raises(ValidationError): diff --git a/tests/unit/sampletones_player/registers/test_triangle.py b/tests/unit/sampletones_player/registers/test_triangle.py index 55e592af9..986284a29 100644 --- a/tests/unit/sampletones_player/registers/test_triangle.py +++ b/tests/unit/sampletones_player/registers/test_triangle.py @@ -3,12 +3,21 @@ from sampletones_core.configs import Config from sampletones_core.constants.enums import ChannelName from sampletones_core.constants.general import MAX_VOLUME -from sampletones_core.generators.implementation.triangle import TriangleGenerator +from sampletones_core.generators.implementation.triangle import ( + TriangleGenerator, +) from sampletones_core.instructions import TriangleInstruction from sampletones_player.registers.pulse import PulseRegisters from sampletones_player.registers.triangle import TriangleRegisters -from sampletones_player.specification.registers import MAX_REGISTER_VALUE, TIMER_HIGH_SHIFT -from tests.suite.player import PLAYER_REFERENCE_PITCH, PLAYER_TIMER_TABLE, sounding_pulse +from sampletones_player.specification.registers import ( + MAX_REGISTER_VALUE, + TIMER_HIGH_SHIFT, +) +from tests.suite.player import ( + PLAYER_REFERENCE_PITCH, + PLAYER_TIMER_TABLE, + sounding_pulse, +) def sounding_triangle() -> TriangleInstruction: @@ -21,7 +30,11 @@ class TestTriangleTickRecord: def test_a_tick_states_the_counter_then_timer(self) -> None: registers = TriangleRegisters.from_instructions([sounding_triangle()], PLAYER_TIMER_TABLE)[0] timer = PLAYER_TIMER_TABLE[PLAYER_REFERENCE_PITCH] - assert registers.values == (0xFF, timer & MAX_REGISTER_VALUE, timer >> TIMER_HIGH_SHIFT) + assert registers.values == ( + 0xFF, + timer & MAX_REGISTER_VALUE, + timer >> TIMER_HIGH_SHIFT, + ) class TestTriangleRegisters: @@ -36,14 +49,23 @@ def test_sounding_tick_reloads_the_counter_fully(self) -> None: assert registers[0].linear_counter == 0xFF def test_resting_tick_reloads_the_counter_to_zero(self) -> None: - instructions = [sounding_triangle(), TriangleInstruction.null_instruction()] + instructions = [ + sounding_triangle(), + TriangleInstruction.null_instruction(), + ] registers = TriangleRegisters.from_instructions(instructions, PLAYER_TIMER_TABLE) assert registers[1].linear_counter == 0x80 def test_rest_keeps_the_timer(self) -> None: - instructions = [sounding_triangle(), TriangleInstruction.null_instruction()] + instructions = [ + sounding_triangle(), + TriangleInstruction.null_instruction(), + ] sounding, resting = TriangleRegisters.from_instructions(instructions, PLAYER_TIMER_TABLE)[:2] - assert (resting.timer_low, resting.timer_high) == (sounding.timer_low, sounding.timer_high) + assert (resting.timer_low, resting.timer_high) == ( + sounding.timer_low, + sounding.timer_high, + ) def test_triangle_shares_the_pulse_timer(self) -> None: triangle = TriangleRegisters.from_instructions([sounding_triangle()], PLAYER_TIMER_TABLE) @@ -51,7 +73,10 @@ def test_triangle_shares_the_pulse_timer(self) -> None: [sounding_pulse(PLAYER_REFERENCE_PITCH, MAX_VOLUME, 0)], PLAYER_TIMER_TABLE, ) - assert (triangle[0].timer_low, triangle[0].timer_high) == (pulse[0].timer_low, pulse[0].timer_high) + assert (triangle[0].timer_low, triangle[0].timer_high) == ( + pulse[0].timer_low, + pulse[0].timer_high, + ) class TestTriangleBend: @@ -85,4 +110,7 @@ def test_a_rest_holds_the_bent_divider(self) -> None: TriangleInstruction.null_instruction(), ] sounding, resting = TriangleRegisters.from_instructions(instructions, PLAYER_TIMER_TABLE)[:2] - assert (resting.timer_low, resting.timer_high) == (sounding.timer_low, sounding.timer_high) + assert (resting.timer_low, resting.timer_high) == ( + sounding.timer_low, + sounding.timer_high, + ) diff --git a/tests/unit/sampletones_player/specification/test_planes.py b/tests/unit/sampletones_player/specification/test_planes.py new file mode 100644 index 000000000..d9fec442b --- /dev/null +++ b/tests/unit/sampletones_player/specification/test_planes.py @@ -0,0 +1,74 @@ +from typing import Final + +import pytest + +from sampletones_core.constants.enums import ( + ALL_CHANNELS, + TONE_CHANNELS, + ChannelName, +) +from sampletones_player.compression.pitch import PITCH_COUNT +from sampletones_player.compression.planes.order import PlaneOrder +from sampletones_player.specification.planes import ( + PLANE_COUNT, + PLANE_NAMES, + PLANES, + SILENT_PITCH_INDEX, + PlaneRole, + channel_indices, + plane_index, +) + +PLANES_A_CHANNEL_SOUNDS_WITH: Final[int] = 2 + + +class TestOneTableStatesEveryPlaneASongBlockWrites: + """The song block writes its planes in one order, and the table is where that order is stated.""" + + def test_every_channel_names_what_it_sounds(self) -> None: + for channel in ChannelName.items(): + roles = {PLANES[index].role for index in channel_indices(channel)} + + assert PlaneRole.VALUE in roles + + def test_a_control_belongs_to_each_channel_whose_timbre_turns_over( + self, + ) -> None: + timbred = {plane.channel for plane in PLANES if plane.role is PlaneRole.CONTROL} + + assert timbred == ALL_CHANNELS - {ChannelName.TRIANGLE} + + def test_the_triangle_names_its_silence_in_the_pitch_it_stands_at( + self, + ) -> None: + triangle = PLANES[plane_index(ChannelName.TRIANGLE, PlaneRole.VALUE)] + + assert triangle.seeded == SILENT_PITCH_INDEX + assert triangle.seeded >= PITCH_COUNT + + def test_a_bend_belongs_to_each_channel_whose_divider_moves(self) -> None: + bending = {PLANES[index].channel for index, plane in enumerate(PLANES) if plane.spans_flagged_ticks} + + assert bending == TONE_CHANNELS + + def test_a_plane_is_named_by_the_channel_and_the_part_it_carries( + self, + ) -> None: + assert PLANES[plane_index(ChannelName.NOISE, PlaneRole.VALUE)].name == "noise_value" + + def test_the_table_names_each_plane_once(self) -> None: + assert len(set(PLANE_NAMES)) == PLANE_COUNT + + def test_a_channel_writing_no_plane_for_a_role_is_refused(self) -> None: + with pytest.raises(ValueError, match="writes no bend plane"): + plane_index(ChannelName.NOISE, PlaneRole.BEND) + + +class TestTheStreamsStandUnderTheNamesTheTableStates: + """A stream is reached by its own name, and the names are the table's — a test keeps them one.""" + + def test_the_streams_carry_the_names_the_table_states(self) -> None: + assert PlaneOrder._fields == PLANE_NAMES + + def test_the_noise_channel_reads_no_bend(self) -> None: + assert len(channel_indices(ChannelName.NOISE)) == PLANES_A_CHANNEL_SOUNDS_WITH diff --git a/tests/unit/sampletones_player/test_builder.py b/tests/unit/sampletones_player/test_builder.py index a45e4a72e..beebcfa6f 100644 --- a/tests/unit/sampletones_player/test_builder.py +++ b/tests/unit/sampletones_player/test_builder.py @@ -62,7 +62,9 @@ def melody() -> List[InstructionUnion]: return [sounding_pulse(PLAYER_REFERENCE_PITCH, PLAYER_FULL_VOLUME, 0) for _ in range(SOUNDING_TICKS)] -def one_channel(generator: ChannelName) -> Dict[ChannelName, List[InstructionUnion]]: +def one_channel( + generator: ChannelName, +) -> Dict[ChannelName, List[InstructionUnion]]: return {generator: melody()} @@ -89,25 +91,39 @@ def bass(*, loop: bool) -> InstrumentExport: class TestStreamsFromInstructions: """The four channels encoded together, each through the encoder its own type names.""" - def test_a_sounding_channel_carries_a_tick_per_instruction_and_a_release(self) -> None: + def test_a_sounding_channel_carries_a_tick_per_instruction_and_a_release( + self, + ) -> None: streams = streams_from_instructions(one_channel(ChannelName.PULSE1), PLAYER_TIMER_TABLE) assert len(streams.pulse1) == SOUNDING_TICKS + 1 def test_a_channel_describing_no_frame_carries_a_single_tick(self) -> None: streams = streams_from_instructions(one_channel(ChannelName.PULSE1), PLAYER_TIMER_TABLE) - assert (len(streams.pulse2), len(streams.triangle), len(streams.noise)) == (1, 1, 1) + assert ( + len(streams.pulse2), + len(streams.triangle), + len(streams.noise), + ) == (1, 1, 1) def test_a_pitch_reaches_the_timer_the_table_states(self) -> None: streams = streams_from_instructions(one_channel(ChannelName.PULSE1), PLAYER_TIMER_TABLE) timer = PLAYER_TIMER_TABLE[PLAYER_REFERENCE_PITCH] - assert (streams.pulse1[0].timer_low, streams.pulse1[0].timer_high) == (timer & 0xFF, timer >> 8) + assert (streams.pulse1[0].timer_low, streams.pulse1[0].timer_high) == ( + timer & 0xFF, + timer >> 8, + ) def test_each_channel_reads_its_own_stream(self) -> None: instructions: Dict[ChannelName, List[InstructionUnion]] = { ChannelName.PULSE2: melody(), ChannelName.TRIANGLE: [TriangleInstruction(on=True, pitch=BASS_PITCH)], ChannelName.NOISE: [ - NoiseInstruction(on=True, period=NOISE_PERIOD, volume=NOISE_VOLUME, short=False), + NoiseInstruction( + on=True, + period=NOISE_PERIOD, + volume=NOISE_VOLUME, + short=False, + ), ], } streams = streams_from_instructions(instructions, PLAYER_TIMER_TABLE) @@ -115,7 +131,9 @@ def test_each_channel_reads_its_own_stream(self) -> None: assert streams.triangle[0].linear_counter == TRIANGLE_COUNTER_CONTROL | TRIANGLE_SOUNDING_RELOAD assert streams.noise[0].control & 0x0F == NOISE_VOLUME - def test_a_channel_holding_another_channels_instructions_raises(self) -> None: + def test_a_channel_holding_another_channels_instructions_raises( + self, + ) -> None: instructions: Dict[ChannelName, List[InstructionUnion]] = {ChannelName.TRIANGLE: melody()} with pytest.raises(TypeError): streams_from_instructions(instructions, PLAYER_TIMER_TABLE) @@ -131,7 +149,9 @@ def test_the_song_lasts_the_ticks_its_longest_channel_covers(self) -> None: == SOUNDING_TICKS + 1 ) - def test_the_schedule_follows_the_rate_the_reconstruction_was_built_at(self) -> None: + def test_the_schedule_follows_the_rate_the_reconstruction_was_built_at( + self, + ) -> None: reconstruction = player_reconstruction(one_channel(ChannelName.PULSE1), HALF_RATE_FREQUENCY) song = song_from_reconstruction(reconstruction, loop_tick=None, scheme=CompressionScheme.SEARCH) assert song.schedule == PlaySchedule.from_parameters(HALF_RATE_FREQUENCY) @@ -143,13 +163,22 @@ def test_the_song_carries_the_loop_it_is_given(self) -> None: def test_a_loop_beyond_the_songs_ticks_raises(self) -> None: reconstruction = player_reconstruction(one_channel(ChannelName.PULSE1), NTSC_FREQUENCY) with pytest.raises(ValueError): - song_from_reconstruction(reconstruction, loop_tick=SOUNDING_TICKS + 1, scheme=CompressionScheme.SEARCH) - - def test_the_timers_come_from_the_reconstructions_own_configuration(self) -> None: + song_from_reconstruction( + reconstruction, + loop_tick=SOUNDING_TICKS + 1, + scheme=CompressionScheme.SEARCH, + ) + + def test_the_timers_come_from_the_reconstructions_own_configuration( + self, + ) -> None: reconstruction = player_reconstruction(one_channel(ChannelName.PULSE1), NTSC_FREQUENCY) song = song_from_reconstruction(reconstruction, loop_tick=None, scheme=CompressionScheme.SEARCH) timer = get_timer_table(reconstruction.config.tuning)[PLAYER_REFERENCE_PITCH] - assert (song.streams.pulse1[0].timer_low, song.streams.pulse1[0].timer_high) == (timer & 0xFF, timer >> 8) + assert ( + song.streams.pulse1[0].timer_low, + song.streams.pulse1[0].timer_high, + ) == (timer & 0xFF, timer >> 8) def test_a_retuned_reconstruction_plays_retuned_timers(self) -> None: """The console reaches a pitch through a timer, so a reconstruction built against another @@ -164,9 +193,14 @@ def test_a_retuned_reconstruction_plays_retuned_timers(self) -> None: song = song_from_reconstruction(retuned, loop_tick=None, scheme=CompressionScheme.SEARCH) assert timer > PLAYER_TIMER_TABLE[PLAYER_REFERENCE_PITCH] - assert (song.streams.pulse1[0].timer_low, song.streams.pulse1[0].timer_high) == (timer & 0xFF, timer >> 8) + assert ( + song.streams.pulse1[0].timer_low, + song.streams.pulse1[0].timer_high, + ) == (timer & 0xFF, timer >> 8) - def test_a_reconstruction_describing_no_frame_plays_one_resting_tick(self) -> None: + def test_a_reconstruction_describing_no_frame_plays_one_resting_tick( + self, + ) -> None: reconstruction = player_reconstruction({ChannelName.PULSE1: [silent_pulse()]}, NTSC_FREQUENCY) song = song_from_reconstruction(reconstruction, loop_tick=None, scheme=CompressionScheme.SEARCH) assert song.ticks == 1 @@ -189,7 +223,9 @@ def test_a_slice_reaches_its_own_channel(self) -> None: instructions = instructions_from_instruments((lead(loop=False), bass(loop=False))) assert set(instructions) == {ChannelName.PULSE1, ChannelName.TRIANGLE} - def test_a_slice_reads_back_as_the_instruction_its_channel_sounds(self) -> None: + def test_a_slice_reads_back_as_the_instruction_its_channel_sounds( + self, + ) -> None: instructions = instructions_from_instruments((lead(loop=False), bass(loop=False))) assert all(isinstance(item, PulseInstruction) for item in instructions[ChannelName.PULSE1]) assert all(isinstance(item, TriangleInstruction) for item in instructions[ChannelName.TRIANGLE]) @@ -219,15 +255,27 @@ def test_a_request_carrying_no_slice_ends_the_song(self) -> None: class TestSongFromSample: """An export request read as the song the console plays it as.""" - def test_every_slice_sounds_on_the_channel_it_was_reconstructed_for(self) -> None: - song = sample_song(player_sample("demo", (lead(loop=False), bass(loop=False)), nes_frequency=NTSC_FREQUENCY)) + def test_every_slice_sounds_on_the_channel_it_was_reconstructed_for( + self, + ) -> None: + song = sample_song( + player_sample( + "demo", + (lead(loop=False), bass(loop=False)), + nes_frequency=NTSC_FREQUENCY, + ) + ) assert len(song.streams.pulse1) == SOUNDING_TICKS + 1 assert song.streams.triangle[0].linear_counter == TRIANGLE_COUNTER_CONTROL | TRIANGLE_SOUNDING_RELOAD assert set(song.streams.pulse2) == {channel_registers(ChannelName.PULSE2, {}, PLAYER_TIMER_TABLE)[0]} def test_a_slice_on_a_channel_the_song_leaves_out_rests(self) -> None: song = song_from_sample( - player_sample("demo", (lead(loop=False), bass(loop=False)), nes_frequency=NTSC_FREQUENCY), + player_sample( + "demo", + (lead(loop=False), bass(loop=False)), + nes_frequency=NTSC_FREQUENCY, + ), channels=frozenset({ChannelName.PULSE1}), loop_tick=None, scheme=CompressionScheme.SEARCH, @@ -251,16 +299,29 @@ def test_the_song_carries_the_loop_it_is_given(self) -> None: def test_the_timers_come_from_the_tuning_the_request_carries(self) -> None: song = sample_song(player_sample("demo", (lead(loop=False),), nes_frequency=NTSC_FREQUENCY)) timer = get_timer_table(Tuning())[PLAYER_REFERENCE_PITCH] - assert (song.streams.pulse1[0].timer_low, song.streams.pulse1[0].timer_high) == (timer & 0xFF, timer >> 8) + assert ( + song.streams.pulse1[0].timer_low, + song.streams.pulse1[0].timer_high, + ) == (timer & 0xFF, timer >> 8) def test_a_retuned_request_plays_retuned_timers(self) -> None: """The console reaches a pitch through a timer, so a request built against another concert pitch plays the divider that concert pitch names. """ tuning = Tuning(a4_frequency=RETUNED_A4_FREQUENCY) - song = sample_song(player_sample("demo", (lead(loop=False),), nes_frequency=NTSC_FREQUENCY, tuning=tuning)) + song = sample_song( + player_sample( + "demo", + (lead(loop=False),), + nes_frequency=NTSC_FREQUENCY, + tuning=tuning, + ) + ) timer = get_timer_table(tuning)[PLAYER_REFERENCE_PITCH] - assert (song.streams.pulse1[0].timer_low, song.streams.pulse1[0].timer_high) == (timer & 0xFF, timer >> 8) + assert ( + song.streams.pulse1[0].timer_low, + song.streams.pulse1[0].timer_high, + ) == (timer & 0xFF, timer >> 8) ROWS_PER_PATTERN: Final[int] = 4 @@ -300,7 +361,9 @@ def project_song(project: Project, loop_tick: Optional[int]) -> Song: class TestSongFromProject: """A whole project reaching the console as the four streams the driver plays.""" - def test_the_song_lasts_the_ticks_the_projects_groove_gives_its_rows(self) -> None: + def test_the_song_lasts_the_ticks_the_projects_groove_gives_its_rows( + self, + ) -> None: project = drum_project() song = project_song(project, loop_tick=None) assert song.ticks == SongTiming.from_project(project).frame_tick(project.song.order_length()) @@ -317,7 +380,10 @@ def test_every_channel_carries_the_songs_whole_length(self) -> None: def test_a_pitch_reaches_the_timer_the_tuning_names(self) -> None: song = project_song(drum_project(), loop_tick=None) timer = get_timer_table(Tuning())[PLAYER_REFERENCE_PITCH] - assert (song.streams.pulse1[0].timer_low, song.streams.pulse1[0].timer_high) == (timer & 0xFF, timer >> 8) + assert ( + song.streams.pulse1[0].timer_low, + song.streams.pulse1[0].timer_high, + ) == (timer & 0xFF, timer >> 8) def test_the_song_carries_the_loop_it_is_given(self) -> None: song = project_song(drum_project(), loop_tick=SONG_START) @@ -355,7 +421,12 @@ def test_the_song_writes_the_registers_the_hardest_scheme_writes(self, scheme: C def test_a_harder_scheme_takes_no_more_room(self) -> None: project = drum_project() sizes = [ - song_from_project(project, channels=ALL_CHANNELS, loop_tick=SONG_START, scheme=scheme).planes.size + song_from_project( + project, + channels=ALL_CHANNELS, + loop_tick=SONG_START, + scheme=scheme, + ).planes.size for scheme in CompressionScheme ] assert sizes == sorted(sizes, reverse=True) diff --git a/tests/unit/sampletones_player/test_instrument_song.py b/tests/unit/sampletones_player/test_instrument_song.py index 769aad1da..1d60bc17c 100644 --- a/tests/unit/sampletones_player/test_instrument_song.py +++ b/tests/unit/sampletones_player/test_instrument_song.py @@ -19,7 +19,9 @@ def _project() -> Project: instrument = Instrument( name="Lead", envelopes=InstrumentEnvelopes( - volume=Envelope(items=VOLUME), arpeggio=Envelope(items=(0, 4, 7)), duty_cycle=Envelope(items=(1,)) + volume=Envelope(items=VOLUME), + arpeggio=Envelope(items=(0, 4, 7)), + duty_cycle=Envelope(items=(1,)), ), ) project = Project.create(title="Demo", rows_per_pattern=ROWS_PER_PATTERN) @@ -34,17 +36,34 @@ class TestAnInstrumentReachesTheConsole: """The player reads the same walk the sequencer plays, so an instrument needs nothing of its own.""" def test_a_project_holding_an_instrument_compiles(self) -> None: - song = song_from_project(_project(), channels=ALL_CHANNELS, loop_tick=None, scheme=CompressionScheme.SEARCH) + song = song_from_project( + _project(), + channels=ALL_CHANNELS, + loop_tick=None, + scheme=CompressionScheme.SEARCH, + ) assert song.planes.ticks > 0 - def test_the_compiled_song_sounds_the_instrument_on_the_channel_it_was_placed_on(self) -> None: - song = song_from_project(_project(), channels=ALL_CHANNELS, loop_tick=None, scheme=CompressionScheme.SEARCH) + def test_the_compiled_song_sounds_the_instrument_on_the_channel_it_was_placed_on( + self, + ) -> None: + song = song_from_project( + _project(), + channels=ALL_CHANNELS, + loop_tick=None, + scheme=CompressionScheme.SEARCH, + ) levels = [registers.control & VOLUME_NIBBLE for registers in song.streams.pulse1[: len(VOLUME)]] assert levels == list(VOLUME) def test_the_channels_it_was_not_placed_on_stay_silent(self) -> None: - song = song_from_project(_project(), channels=ALL_CHANNELS, loop_tick=None, scheme=CompressionScheme.SEARCH) + song = song_from_project( + _project(), + channels=ALL_CHANNELS, + loop_tick=None, + scheme=CompressionScheme.SEARCH, + ) assert {registers.control & VOLUME_NIBBLE for registers in song.streams.pulse2} == {0} diff --git a/tests/unit/sampletones_player/test_song.py b/tests/unit/sampletones_player/test_song.py index 133dc586a..8e26e2a05 100644 --- a/tests/unit/sampletones_player/test_song.py +++ b/tests/unit/sampletones_player/test_song.py @@ -36,7 +36,11 @@ def test_a_song_may_stand_without_a_loop(self) -> None: def test_a_loop_at_the_songs_length_raises(self) -> None: with pytest.raises(ValidationError, match="loop_tick must lie within"): - player_song(resting_streams((SOUNDING, OCTAVE_UP)), NTSC_FREQUENCY, loop_tick=2) + player_song( + resting_streams((SOUNDING, OCTAVE_UP)), + NTSC_FREQUENCY, + loop_tick=2, + ) def test_a_negative_loop_raises(self) -> None: with pytest.raises(ValidationError, match="loop_tick must lie within"): @@ -70,10 +74,26 @@ def song(self) -> Song: ) test_cases: Tuple[TestCase, ...] = ( - TestCase(name="stops", loop_tick=None, expected=(0, 1, 2, 3, None, None, None, None)), - TestCase(name="repeats-from-the-start", loop_tick=0, expected=(0, 1, 2, 3, 0, 1, 2, 3)), - TestCase(name="repeats-from-the-middle", loop_tick=2, expected=(0, 1, 2, 3, 2, 3, 2, 3)), - TestCase(name="repeats-one-tick", loop_tick=3, expected=(0, 1, 2, 3, 3, 3, 3, 3)), + TestCase( + name="stops", + loop_tick=None, + expected=(0, 1, 2, 3, None, None, None, None), + ), + TestCase( + name="repeats-from-the-start", + loop_tick=0, + expected=(0, 1, 2, 3, 0, 1, 2, 3), + ), + TestCase( + name="repeats-from-the-middle", + loop_tick=2, + expected=(0, 1, 2, 3, 2, 3, 2, 3), + ), + TestCase( + name="repeats-one-tick", + loop_tick=3, + expected=(0, 1, 2, 3, 3, 3, 3, 3), + ), ) @pytest.mark.parametrize("test_case", test_cases, ids=lambda test_case: test_case.label) @@ -89,9 +109,24 @@ def test_every_tick_played_lies_within_the_song(self, test_case: TestCase) -> No assert all(0 <= tick < song.ticks for tick in played if tick is not None) def test_the_song_lasts_as_long_as_its_streams(self) -> None: - song = player_song(resting_streams((SOUNDING, OCTAVE_UP)), NTSC_FREQUENCY, loop_tick=None) + song = player_song( + resting_streams((SOUNDING, OCTAVE_UP)), + NTSC_FREQUENCY, + loop_tick=None, + ) assert song.ticks == song.streams.ticks def test_a_slow_stream_holds_its_tick_between_calls(self) -> None: - song = player_song(resting_streams((SOUNDING, OCTAVE_UP, RESTING, RESTING)), 30, loop_tick=None) - assert tuple(song.tick_at(play_call) for play_call in range(6)) == (0, 0, 1, 1, 2, 2) + song = player_song( + resting_streams((SOUNDING, OCTAVE_UP, RESTING, RESTING)), + 30, + loop_tick=None, + ) + assert tuple(song.tick_at(play_call) for play_call in range(6)) == ( + 0, + 0, + 1, + 1, + 2, + 2, + ) diff --git a/tests/unit/sampletones_tools/codec/study/test_accounting.py b/tests/unit/sampletones_tools/codec/study/test_accounting.py index f22839b39..3a11c4e33 100644 --- a/tests/unit/sampletones_tools/codec/study/test_accounting.py +++ b/tests/unit/sampletones_tools/codec/study/test_accounting.py @@ -4,7 +4,7 @@ import pytest from sampletones_player.compression.dictionary.phrase import Phrase -from sampletones_player.compression.dictionary.table import phrase_table +from sampletones_player.compression.dictionary.table import PhraseTable, phrase_table from sampletones_player.compression.encode import emit from sampletones_player.compression.planes.order import PlaneOrder from sampletones_player.compression.tokens.hold import HoldToken @@ -12,8 +12,9 @@ from sampletones_player.compression.tokens.phrase import PhraseToken from sampletones_player.compression.tokens.types import TokenUnion from sampletones_player.specification.compression import MAX_HOLD_TICKS, TokenTag +from sampletones_player.specification.planes import PLANES, PlaneRole from sampletones_tools.codec.study.accounting.coincident import coincident_starts -from sampletones_tools.codec.study.accounting.dictionary import default_counts, plateaus +from sampletones_tools.codec.study.accounting.dictionary import plateaus from sampletones_tools.codec.study.accounting.finding import Finding from sampletones_tools.codec.study.accounting.pairs import ramps_in_literals, set_holds from sampletones_tools.codec.study.accounting.runs import ramps, runs @@ -32,12 +33,12 @@ class TestReadTokensReadsBackWhatEmitWrote: tokens: Final[Tuple[TokenUnion, ...]] = ( HoldToken(ticks=3), LiteralToken(values=b"\x01\x02"), - PhraseToken(phrase_id=2, ticks=4, transpose=0), - PhraseToken(phrase_id=ESCAPED_PHRASE_ID, ticks=2, transpose=3), + PhraseToken(phrase_id=2, ticks=4, transpose=0, default=False), + PhraseToken(phrase_id=ESCAPED_PHRASE_ID, ticks=2, transpose=3, default=False), ) def test_the_tokens_come_back_in_order(self) -> None: - read = read_tokens(emit(self.tokens)) + read = read_tokens(emit(self.tokens), DICTIONARY) assert [token.tag for token in read] == [ TokenTag.HOLD, @@ -50,7 +51,7 @@ def test_the_tokens_come_back_in_order(self) -> None: assert [token.size for token in read] == [token.size for token in self.tokens] def test_the_operands_come_back(self) -> None: - read = read_tokens(emit(self.tokens)) + read = read_tokens(emit(self.tokens), DICTIONARY) assert [token.payload for token in read] == [b"", b"\x01\x02", b"", b""] assert [token.phrase_id for token in read] == [None, None, 2, ESCAPED_PHRASE_ID] @@ -95,6 +96,9 @@ def test_the_constant_step_runs_are_found(self, test_case: TestCase) -> None: assert ramps(test_case.data) == test_case.expected +DICTIONARY: Final[PhraseTable] = phrase_table(tuple(Phrase(body=bytes((value,))) for value in range(4))) + + class TestHoldChains(BaseTestSuite): """A wide hold pays two bytes for up to sixty-four full holds, and one for the ticks left.""" @@ -127,7 +131,7 @@ class TestCase(BaseRegularTestCase): def test_the_chain_is_priced_against_wide_holds(self, test_case: TestCase) -> None: stream = emit([HoldToken(ticks=ticks) for ticks in test_case.holds]) - assert hold_chains(read_tokens(stream)) == test_case.expected + assert hold_chains(read_tokens(stream, DICTIONARY)) == test_case.expected def test_a_literal_breaks_the_chain(self) -> None: stream = emit( @@ -139,7 +143,7 @@ def test_a_literal_breaks_the_chain(self) -> None: ] ) - assert hold_chains(read_tokens(stream)) == Finding(2, 0) + assert hold_chains(read_tokens(stream, DICTIONARY)) == Finding(2, 0) class TestSetHolds(BaseTestSuite): @@ -178,7 +182,7 @@ class TestCase(BaseRegularTestCase): @pytest.mark.parametrize("test_case", test_cases, ids=lambda test_case: test_case.label) def test_the_pairs_are_priced_at_two_bytes(self, test_case: TestCase) -> None: - assert set_holds(read_tokens(emit(list(test_case.tokens)))) == test_case.expected + assert set_holds(read_tokens(emit(list(test_case.tokens)), DICTIONARY)) == test_case.expected class TestRampsInLiterals(BaseTestSuite): @@ -196,10 +200,13 @@ class TestCase(BaseRegularTestCase): @pytest.mark.parametrize("test_case", test_cases, ids=lambda test_case: test_case.label) def test_the_ramps_are_priced_at_three_bytes(self, test_case: TestCase) -> None: - assert ramps_in_literals(read_tokens(emit([LiteralToken(values=test_case.values)]))) == test_case.expected + assert ( + ramps_in_literals(read_tokens(emit([LiteralToken(values=test_case.values)]), DICTIONARY)) + == test_case.expected + ) def test_a_ramp_inside_a_hold_is_none(self) -> None: - assert ramps_in_literals(read_tokens(emit([HoldToken(ticks=5)]))) == Finding(0, 0) + assert ramps_in_literals(read_tokens(emit([HoldToken(ticks=5)]), DICTIONARY)) == Finding(0, 0) def _starting_at(ticks: Sequence[int]) -> Tuple[ReadToken, ...]: @@ -218,13 +225,17 @@ def _starting_at(ticks: Sequence[int]) -> Tuple[ReadToken, ...]: class TestCoincidentStarts: + """Every channel carrying a timbre shares its first tick, and pulse one shares a second.""" + def test_a_channel_counts_the_ticks_both_its_planes_start_a_token_on(self) -> None: tokens: Dict[str, Tuple[ReadToken, ...]] = {name: _starting_at((0,)) for name in PlaneOrder.names()} tokens["pulse1_control"] = _starting_at((0, 5)) tokens["pulse1_value"] = _starting_at((0, 5, 9)) tokens["noise_value"] = _starting_at((0, 7)) + timbred = sum(1 for plane in PLANES if plane.role is PlaneRole.CONTROL) + shared = timbred + 1 - assert coincident_starts(tokens) == Finding(10, 5) + assert coincident_starts(tokens) == Finding(2 * shared, shared) class TestPlateausInBodies: @@ -232,21 +243,3 @@ def test_the_repeats_beyond_a_runs_first_byte_are_targeted(self) -> None: table = phrase_table((Phrase(body=b"\x01\x01\x01\x02\x02"), Phrase(body=b"\x03\x04"))) assert plateaus(table) == Finding(3, 1) - - -class TestDefaultCounts: - def test_the_modal_count_of_each_phrase_spares_its_bytes_but_one(self) -> None: - table = phrase_table((Phrase(body=b"\x01\x02"), Phrase(body=b"\x03\x04"))) - tokens = read_tokens( - emit( - [ - PhraseToken(phrase_id=0, ticks=4, transpose=0), - PhraseToken(phrase_id=0, ticks=4, transpose=0), - PhraseToken(phrase_id=0, ticks=4, transpose=2), - PhraseToken(phrase_id=0, ticks=6, transpose=0), - PhraseToken(phrase_id=1, ticks=2, transpose=0), - ] - ) - ) - - assert default_counts(table, tokens) == Finding(5, 2) diff --git a/tests/unit/sampletones_tools/codec/study/test_layouts.py b/tests/unit/sampletones_tools/codec/study/test_layouts.py index 44d4777d1..57ac13a04 100644 --- a/tests/unit/sampletones_tools/codec/study/test_layouts.py +++ b/tests/unit/sampletones_tools/codec/study/test_layouts.py @@ -10,14 +10,13 @@ from sampletones_core.project.voices.instrument import Instrument from sampletones_core.timers.utils import get_timer_table from sampletones_player.builder import streams_from_instructions -from sampletones_player.compression.absent import is_absent from sampletones_player.compression.pitch import PitchTable -from sampletones_player.compression.planes.channel import TonePlanes from sampletones_player.compression.planes.separate import channel_planes, planes_from_streams from sampletones_player.compression.seeds import phrases_from_project from sampletones_player.registers.channel import channel_registers from sampletones_player.specification.binary import unsigned_byte from sampletones_player.specification.compression import BEND_FLAG +from sampletones_player.specification.planes import PLANES from sampletones_shared.music import Tuning from sampletones_tools.codec.study.corpus.notes import channel_notes, song_notes from sampletones_tools.codec.study.corpus.slices import project_slices @@ -56,11 +55,12 @@ def coarse_frame(pitch: int, coarse: int) -> List[InstructionUnion]: return [PulseInstruction(on=True, pitch=pitch, volume=PLAYER_FULL_VOLUME, duty_cycle=0, coarse_detune=coarse)] -def pulse_planes(instructions: Sequence[InstructionUnion]) -> TonePlanes: +VALUE_PLANE: Final[int] = 1 + + +def pulse_planes(instructions: Sequence[InstructionUnion]) -> Tuple[bytes, ...]: registers = channel_registers(ChannelName.PULSE1, {ChannelName.PULSE1: instructions}, TIMER_TABLE) - planes = channel_planes(ChannelName.PULSE1, registers, PITCHES) - assert isinstance(planes, TonePlanes) - return planes + return channel_planes(ChannelName.PULSE1, registers, PITCHES) def written(instructions: Sequence[InstructionUnion], layout: PlaneLayout) -> Tuple[bytes, bytes, bytes]: @@ -115,7 +115,7 @@ def test_the_named_flagged_notes_layout_is_the_production_separation(self) -> No *coarse_frame(PLAYER_REFERENCE_PITCH, BEYOND_A_BYTE), ] planes = pulse_planes(instructions) - assert written(instructions, PRODUCTION) == planes.ordered + assert written(instructions, PRODUCTION) == planes def test_a_named_note_keeps_a_bend_past_halfway(self) -> None: instructions = bent_frames(PLAYER_REFERENCE_PITCH, (PAST_HALFWAY,)) @@ -163,7 +163,7 @@ def test_an_unbent_channel_holds_no_bend_and_no_flag(self, form: BendForm) -> No instructions = bent_frames(PLAYER_REFERENCE_PITCH, (0, 0, 0)) _, value, bend = written(instructions, PlaneLayout(Anchor.NEAREST, form)) assert bend == b"" - assert value == pulse_planes(instructions).value + assert value == pulse_planes(instructions)[VALUE_PLANE] class TestALayoutsSeeds: @@ -192,6 +192,7 @@ def test_an_absent_plane_costs_no_stream(self, layout: PlaneLayout) -> None: song = study_song(bent_frames(PLAYER_REFERENCE_PITCH, (0, 3, 0))) planes = layout_planes(song, layout) streams = encode_layout(song, layout).streams - assert any(is_absent(plane) for plane in planes) - assert all(size == 0 for plane, size in zip(planes, streams) if is_absent(plane)) - assert all(size > 0 for plane, size in zip(planes, streams) if not is_absent(plane)) + idle = tuple(plane.idles(played) for plane, played in zip(PLANES, planes, strict=True)) + assert any(idle) + assert all(size == 0 for resting, size in zip(idle, streams) if resting) + assert all(size > 0 for resting, size in zip(idle, streams) if not resting) diff --git a/tests/unit/sampletones_tools/codec/study/test_sandbox.py b/tests/unit/sampletones_tools/codec/study/test_sandbox.py index 84341498a..cd3198cac 100644 --- a/tests/unit/sampletones_tools/codec/study/test_sandbox.py +++ b/tests/unit/sampletones_tools/codec/study/test_sandbox.py @@ -20,8 +20,8 @@ from sampletones_player.specification.compression import ( CHEAP_PHRASE_IDS, MAX_HOLD_TICKS, - PLANE_COUNT, ) +from sampletones_player.specification.planes import NO_BITS, PLANE_COUNT, PLANES, PlaneRole from sampletones_shared.music import Tuning from sampletones_tools.codec.study.corpus.song import SongGroup, StudySong from sampletones_tools.codec.study.sandbox.context import PlaneContext @@ -36,11 +36,13 @@ from sampletones_tools.codec.study.sandbox.tokens import Hold, Literal, Play, SetHold, StudyToken, WideHold from sampletones_tools.codec.study.sandbox.verify import verify_baseline from sampletones_tools.codec.study.variants.production import compress_baseline -from sampletones_tools.codec.study.variants.sandbox import DEFAULT_COUNT_COSTS, GRAMMAR_VARIANTS, GrammarVariant +from sampletones_tools.codec.study.variants.sandbox import GRAMMAR_VARIANTS, GrammarVariant from tests.suite.base import BaseTestSuite from tests.suite.case import BaseRegularTestCase +from tests.suite.player import playable from tests.suite.study import NO_STUDY_SLICES, lowest_notes +SEEDED_SYMBOL: Final[int] = NO_BITS ENTRIES: Final[FrozenSet[int]] = frozenset({STREAM_START}) FIGURES: Final[Tuple[bytes, ...]] = (b"\x0a\x0c\x0f\x0f\x0f\x0f\x0f\x0f", b"\x03\x04\x05\x06") TABLE: Final[PhraseTable] = phrase_table(Phrase(body=body) for body in FIGURES) @@ -54,7 +56,6 @@ SET_HOLD_REALLOCATED: Final[Grammar] = replace( BASELINE_GRAMMAR, set_hold=True, costs=replace(PRODUCTION_COSTS, set_hold=SET_HOLD_BOUND) ) -DEFAULT_COUNTS: Final[Grammar] = replace(BASELINE_GRAMMAR, default_counts=True, costs=DEFAULT_COUNT_COSTS) def _figures_plane(random: Random, ticks: int) -> bytes: @@ -83,19 +84,19 @@ def _dense_plane(random: Random, ticks: int) -> bytes: def _song(ticks: int) -> StudySong: random = Random(SEED) planes: List[bytes] = [] - for plane in range(PLANE_COUNT): - if plane % 3 == 2: + for plane in PLANES: + if plane.spans_flagged_ticks: planes.append(b"") - elif plane % 3 == 0: - planes.append(_runs_plane(random, ticks)) + elif plane.role is PlaneRole.CONTROL: + planes.append(playable(plane, _runs_plane(random, ticks))) else: - planes.append(_figures_plane(random, ticks)) + planes.append(playable(plane, _figures_plane(random, ticks))) return StudySong( name="song", group=SongGroup.PROJECT, source=Path("song.stp"), - planes=SongPlanes.from_order(PlaneOrder.across(planes)), + planes=SongPlanes(planes=PlaneOrder.across(planes)), seeds=tuple(Phrase(body=body) for body in FIGURES), pitches=PitchTable.from_tuning(Tuning()), notes=lowest_notes(ticks), @@ -116,6 +117,7 @@ def _context( boundaries=Boundaries.across(len(plane), ENTRIES), transposition=EVERY_LAYER.transposition, defaults=tuple(defaults), + seeded=SEEDED_SYMBOL, costs=costs, ) @@ -300,7 +302,7 @@ class TestCase(BaseRegularTestCase): label="a phrase played at its default count carries no count", plane=repeated, table=repeated_table, - grammar=DEFAULT_COUNTS, + grammar=BASELINE_GRAMMAR, defaults=(4,), expected=3, ), @@ -308,7 +310,7 @@ class TestCase(BaseRegularTestCase): label="a phrase played at another count carries it", plane=repeated, table=repeated_table, - grammar=DEFAULT_COUNTS, + grammar=BASELINE_GRAMMAR, defaults=(3,), expected=6, ), @@ -328,11 +330,6 @@ def test_a_phrase_beyond_the_cheap_ids_pays_the_escape(self) -> None: assert PRODUCTION_COSTS.phrase(CHEAP_PHRASE_IDS, 0, default=False) == 3 assert PRODUCTION_COSTS.phrase(0, 3, default=False) == 3 - def test_default_counts_halve_the_cheap_ids_and_widen_the_entries(self) -> None: - assert DEFAULT_COUNT_COSTS.phrase(30, 0, default=True) == 1 - assert DEFAULT_COUNT_COSTS.phrase(31, 0, default=True) == 2 - assert DEFAULT_COUNT_COSTS.dictionary(TABLE) == TABLE.size + len(TABLE) - def test_offered_lengths_are_the_longest_alone_or_every_one(self) -> None: assert tuple(offered_lengths(5, least=2, every=False)) == (5,) assert tuple(offered_lengths(5, least=2, every=True)) == (2, 3, 4, 5) @@ -398,4 +395,5 @@ def test_a_plane_holding_zero_throughout_is_absent_under_every_grammar(self, ent encoding = encode_grammar(read, entry.grammar) - assert [encoding.streams[plane] for plane in range(2, PLANE_COUNT, 3)] == [0, 0, 0] + bends = [index for index, plane in enumerate(PLANES) if plane.spans_flagged_ticks] + assert [encoding.streams[index] for index in bends] == [0] * len(bends) diff --git a/tests/unit/sampletones_tools/codec/study/test_variants.py b/tests/unit/sampletones_tools/codec/study/test_variants.py index 3bfdc193f..2fd94787a 100644 --- a/tests/unit/sampletones_tools/codec/study/test_variants.py +++ b/tests/unit/sampletones_tools/codec/study/test_variants.py @@ -13,7 +13,7 @@ from sampletones_player.compression.planes.song import SongPlanes from sampletones_player.compression.tokens.hold import HoldToken from sampletones_player.compression.tokens.literal import LiteralToken -from sampletones_player.specification.compression import PLANE_COUNT +from sampletones_player.specification.planes import PLANE_COUNT from sampletones_shared.music import Tuning from sampletones_tools.codec.study.corpus.song import SongGroup, StudySong from sampletones_tools.codec.study.measure import Measurement, production_encoding @@ -120,6 +120,7 @@ def _measurement( compressed = CompressedPlanes( phrases=phrase_table(()), streams=PlaneOrder.across([stream] * PLANE_COUNT), + loop_entries=(0,) * PLANE_COUNT, ticks=TICKS, ) return Measurement( diff --git a/tests/unit/sampletones_tools/codec/study/test_verdicts.py b/tests/unit/sampletones_tools/codec/study/test_verdicts.py index d7ac7f024..6e2ccff2b 100644 --- a/tests/unit/sampletones_tools/codec/study/test_verdicts.py +++ b/tests/unit/sampletones_tools/codec/study/test_verdicts.py @@ -7,7 +7,8 @@ from sampletones_player.compression.pitch import PitchTable from sampletones_player.compression.planes.order import PlaneOrder from sampletones_player.compression.planes.song import SongPlanes -from sampletones_player.specification.compression import PLANE_COUNT +from sampletones_player.specification.planes import PLANE_COUNT +from sampletones_player.specification.song import SONG_HEADER_SIZE from sampletones_shared.music import Tuning from sampletones_tools.codec.study.corpus.song import SongGroup, StudySong from sampletones_tools.codec.study.measure import Encoding, Measurement @@ -117,6 +118,7 @@ def _measurement( song=song, variant=variant, encoding=Encoding( + header=SONG_HEADER_SIZE, phrases=0, dictionary=0, streams=(streams,) + (0,) * (PLANE_COUNT - 1), @@ -134,6 +136,7 @@ def _variant(name: str, kind: VariantKind) -> Variant: kind=kind, note="", encode=lambda song: Encoding( + header=SONG_HEADER_SIZE, phrases=0, dictionary=0, streams=(), diff --git a/tests/unit/sampletones_tools/player/test_song_include.py b/tests/unit/sampletones_tools/player/test_song_include.py index 9d4cc268d..03248a9c1 100644 --- a/tests/unit/sampletones_tools/player/test_song_include.py +++ b/tests/unit/sampletones_tools/player/test_song_include.py @@ -9,18 +9,33 @@ from sampletones_player.specification.binary import WORD_SIZE from sampletones_player.specification.compression import ( BEND_FLAG, + DEFAULT_COUNT_FLAG, OPCODE_SIZE, + PHRASE_DEFAULT_SIZE, PHRASE_ID_ESCAPE, + PHRASE_ID_MASK, PHRASE_LENGTH_SIZE, PHRASE_TABLE_COUNT_SIZE, PHRASE_TABLE_ENTRY_SIZE, PITCH_INDEX_MASK, - PLANE_COUNT, PLANE_STATE_SIZE, TOKEN_OPERAND_MASK, TOKEN_TAG_MASK, TokenTag, ) +from sampletones_player.specification.planes import ( + COUNT_STEP, + NOISE_CONTROL_FORM, + NOISE_VALUE_FORM, + PLANE_COUNT, + PULSE_CONTROL_FORM, + SILENT_PITCH_INDEX, +) +from sampletones_player.specification.registers import ( + TRIANGLE_COUNTER_CONTROL, + TRIANGLE_SILENT_RELOAD, + TRIANGLE_SOUNDING_RELOAD, +) from sampletones_player.specification.song import ( ABSENT_STREAM, LOOP_ENTRIES_OFFSET, @@ -67,8 +82,22 @@ "PHRASE_TABLE_COUNT_SIZE": PHRASE_TABLE_COUNT_SIZE, "PHRASE_TABLE_ENTRY_SIZE": PHRASE_TABLE_ENTRY_SIZE, "PHRASE_LENGTH_SIZE": PHRASE_LENGTH_SIZE, + "PHRASE_DEFAULT_SIZE": PHRASE_DEFAULT_SIZE, + "DEFAULT_COUNT_FLAG": DEFAULT_COUNT_FLAG, + "PHRASE_ID_MASK": PHRASE_ID_MASK, "PLANE_STATE_SIZE": PLANE_STATE_SIZE, "PLANE_STATE_BYTES": PLANE_COUNT * PLANE_STATE_SIZE, + "COUNT_STEP": COUNT_STEP, + "SILENT_PITCH_INDEX": SILENT_PITCH_INDEX, + "TRIANGLE_COUNTER": TRIANGLE_COUNTER_CONTROL, + "TRIANGLE_SOUNDING": TRIANGLE_SOUNDING_RELOAD, + "TRIANGLE_SILENT": TRIANGLE_SILENT_RELOAD, + "PULSE_CONTROL_MASK": PULSE_CONTROL_FORM.value_mask, + "PULSE_CONTROL_FIXED": PULSE_CONTROL_FORM.value_or, + "NOISE_CONTROL_MASK": NOISE_CONTROL_FORM.value_mask, + "NOISE_CONTROL_FIXED": NOISE_CONTROL_FORM.value_or, + "NOISE_VALUE_MASK": NOISE_VALUE_FORM.value_mask, + "NOISE_VALUE_FIXED": NOISE_VALUE_FORM.value_or, }