RAINBOHz is a research project exploring additive synthesis at large scale, referred to as macro-scale additive synthesis. It operates at the "macro-scale" from two perspectives. It allows large numbers of additive partials to be used, and it allows control over the macro timescale i.e. over a complete musical work (or a significant part thereof) defined by a score.
Scores are written in a domain-specific, declarative language called HzL. This language and its compiler are part of this project. It is expected that practitioners will use a combination of hand-coding, assembly from pre-built component libraries, and LLMs to generate HzL scores.
Some key concepts and goals of the project are listed below. Only a limited number of these goals have been implemented to date.
From an audio / music perspective:
- Provide an interactive environment for sound design, composition and live coding.
- Support authoring based on a language for describing scores.
- Scale to large numbers of additive partials.
- Support polyrhythms / polytempo.
- Support spatial audio.
- Allow manipulation of scores at different levels of abstraction.
- Integrate with DAWs for workflow.
- Do not provide a playable "instrument".
From a computer science perspective:
- Provide a domain-specific, declarative language describing a score.
- Allow composition with reusable components and version control through git.
- Support deployment on single-user workstations, LANs, and public cloud platforms.
- Render large numbers of partials with acceptable latency making use of vertical and horizontal scalability.
- Provide rapid rendering and random-access playback based on mathematical integration of state.
The project is currently in an early stage, moving from a proof-of-concept to the target architecture. The current version should be considered as an emerging research project. Much testing, validation and additional features still remain to be added. Here be dragons. It's not only that "your mileage may vary", it's that the wheels might fall off, or pressing the brake pedal might make the doors open. You have been warned!
In particular the following are large gaps in the current implementation.
- Anti-aliasing
- API / network approach (currently some parts are implemented)
- Horizontal scalability
- Build using GitHub actions
The architecture takes all of these issues into account, but they are simply not yet implemented.
Turn down your headphone / speaker level before trying experiments with RAINBOHz. This software can generate unusual audio signals that could damage your speakers or headphones, and more importantly could damage your hearing. Keep those levels down whilst experimenting!
This project was initiated prior to the widespread use of LLMs for code generation. The first proof-of-concept was written following a largely conventional approach.
The current source code and most of the documentation is primarily AI-generated. The use of LLMs has accelerated development of RAINBOHz, but in its current form not all errors have been found, algorithms may be suboptimal, and code style may be inappropriate. As the project matures, code reviews and refactoring will gradually improve this situation.
Some key artefacts such as the graph database schema, and most of the core HzL language have been coded by hand.
RAINBOHz is designed so that it can be deployed and scaled using a distributed architecture. This has not yet been demonstrated. It is a design concept - albeit one that has been the subject of considerable effort, supported by performance and capacity calculations. The current codebase includes some of the initial components that will make this possible, such as an early version of one of the REST APIs.
RAINBOHz can be deployed on a single computing node, which may be sufficient depending upon the required throughput and the acceptable use of local resources. In its current state, this is the only possible deployment model.
Horizontal scalability is based on sharding of partials over multiple nodes. The sharding is in turn based on dividing the domain-specific language source over the nodes. The exact mechanism for this is not yet designed, but there are some clear approaches such as division by sound entity.
This can be performed by placing all HzL code in a git repository, then using partial clones to divide the model between the nodes. It is likely that the sharding approach will evolve from there.
Each processing node is an independent end-to-end synthesis node. Each node performs the full end-to-end process of compiling HzL to the graph database model, performing the database transformations, performing precomputation, and finally rendering frames of audio. When deployed on a distributed system, this audio may then be written to object storage. A more performant option may be to stream the audio to the next stage in the pipeline.
The next stage makes audio available to clients. It performs summing and anti-aliasing, then makes audio available to clients via a REST API.
The pipeline can be viewed as a chain of components as follows:
Compilation — essentially a software build system.
- The Hertz Language, HzL
- Hertz Language Compiler
- Hertz Language Build System
canonical-form— the compiled, loadable score
Rendering — transformation of the score specification to audio
rendering-engine— loads, transforms, precomputes and rendersengine-db— the one graph database, hosted in process, sharded by partial set- Object storage — WAV objects addressed by URL
Playback — merging the shards on demand.
playback-server— sums per-shard objects and serves audio- Playback Component
- Playback Plug-in
Orchestration — a message bus connecting the components. Kafka is the current target. This exists currently only as a design idea.
See architecture/overview.md for the full
description, and §11 there for why an earlier pipelined design was collapsed
into this one.
C++20 using clang / Coding conventions according the C++ Core Guidelines. Python 3.12
This application is intended as a server-side application running on UNIX-like operating systems and/or cloud computing platforms.
The primary goal is to target cloud platforms. This is likely to target AWS Graviton. Other platforms are also under consideration. This will demonstrate horizontal scalability, and will also leverage SIMD and similar to optimize vertical scalability. There has been some consideration as to the use of GPUs, but the current viewpoint is that the ARM architecture will offer the best cost / performance tradeoff and is easier to work with.
This project is currently moving from its proof-of-concept phase towards a scalable architecture.
| Directory | Contents |
|---|---|
architecture/ |
Target architecture: overview and diagrams |
hzl-compiler/ |
HzL compiler, Python — first version: HzL to Cypher for engine-db |
engine-db/ |
Schema of engine-db, the engine's database and the only one — under construction |
canonical-db/ |
Canonical (practitioner-facing) model — design artefact, not the implementation; out of date |
rendering-engine/ |
Rendering engine, C++ — loads, transforms, precomputes and renders; first version |
playback-client/ |
Playback client, C++ — loops a range of a score through the default audio device; first version |
docs/adr/ |
Architecture Decision Records |
cmake/ |
Shared CMake configuration for the C++ components: warnings, presets and cross toolchains |
scripts/ |
Schema loading, validation, diagrams and comment wrapping, the shared DDL machinery, the native model's mapping check, and the score-forms check |
proof-of-concept/ |
Proofs of concept, each in its own sub-directory, tried before being built into the components. The original sub-directories are frozen as a historical record. Not built |
playground/ |
Musical experiments: pg loops an HzL piece through the engine and reloads it on request. No stability guarantee; may change or vanish at any time |
The developer guide is currently only in place for macOS, but it should be possible to follow similar steps on other operating systems.
Everything is driven from the root Makefile. make help lists the targets.
make docs # regenerate this README, schema diagrams and API docs
make readme # just this README, from docs/readme/
make validate-schemas # check the schemas load into a real Ladybug
make check-schema-comments # show schema comments that need rewrapping
make check-engine-db-mapping # check the native engine-db's types against the schema
make init-engine-db # build a database from the engine schema
make rendering-engine # build the rendering engine
make test-rendering-engine # and run its tests
make hzl-compile # compile an HzL score to Cypher (hzl-compiler/)
make run-hzl-score # ...and render it with the engine
make test-hzl-compiler # the compiler's tests
make lint-hzl-compiler # the compiler's lint, format and type checks
make lock # re-lock Python dependencies after changing them (needs uv)
make audit-python # check the locked Python packages for known vulnerabilitiesPython packages are installed into the root .venv from a hash-checked
lock, requirements-dev.txt, which make lock generates from
requirements-dev.in and hzl-compiler/pyproject.toml. Edit those, run
make lock, and commit the lock with them; never edit the lock by hand. See
ADR 0021.
The C++ components are built with CMake behind those targets, and using CMake
directly works just as well — see rendering-engine/README.md.
The first C++ build compiles Ladybug, and Apache Arrow's Parquet reader,
from source and takes a long time.
make test runs schema validation first, because it takes seconds and needs no
compiler — a schema that does not load makes everything after it moot.
The implementation uses one physical database, engine-db, hosted in process by
rendering-engine. It is the engine's whole database: everything in the
canonical model, plus everything the engine needs in order to determine
dependencies, precompute and render. Its schema is
engine-db/schema/engine-schema.cypher, and it is under construction — it
holds the rendering-model and canonical-model concepts, but not yet the modulator
musical timelines or any dependency information. See
ADR 0006.
engine-db has two implementations of that one schema, and a run uses one
of them (ADR 0028,
proposed): the Ladybug graph database, and a native in-memory model whose
hand-written types are tagged with the part of the schema each holds.
hzlc's tables load natively, and much faster; a .cypher score and
--validate-db need Ladybug. make check-engine-db-mapping fails when the
schema and the native types' tags disagree, so a schema edit is followed by
a look at the native side. See
rendering-engine/README.md.
The .cypher schema file under canonical-db/ is a design artefact. It
describes the practitioner-facing model and is kept loadable, but nothing loads
it; it is a source engine-db draws on, not something the system runs on, and it
is now considerably out of date beside engine-db.
Keeping both loadable is worth it: a model that a real database rejects is a
model with a mistake in it, and make validate-schemas catches that in seconds
— run it to see where the two stand. The databases themselves are disposable,
gitignored, and rebuild in a few seconds.
make init-engine-db # apply the schema (safe to repeat)
make reset-engine-db # destroy and recreate from schemaEach directory also carries init_db.sh and reset_db.sh for use outside make.
Never pipe a .cypher file into lbug directly. It reports errors on
stdout, exits 0 regardless, and silently skips statements when fed a whole
file — so schemas half-load with no diagnostic at all. Use the targets above,
which wrap scripts/load_schema.py. See
ADR 0002.
xcode-select --install
brew install cmake ninja ccache llvm ladybugHomebrew's llvm is keg-only and stays off PATH deliberately; it supplies
clang-tidy, which Apple's clang does not ship. Python 3.12+ is needed for the
schema tooling. brew install uv is needed only to change Python
dependencies (make lock).
The project makes extensive use of graph databases, using Ladybug.
There are two separate things here, and the distinction matters:
- The Homebrew package (
brew install ladybug) provides the interactive shell, for inspecting databases by hand. Convenient, not required. - The embedded library is compiled from pinned source as part of the C++ build. This is what the components actually link against. See ADR 0001.
Keep the two at the same version. The storage format changes between releases,
so a shell older than the linked library may not open a database the library
wrote. brew outdated ladybug against RE_LADYBUG_GIT_TAG in
rendering-engine/cmake/Dependencies.cmake
is the check.
scripts/ladybug_schema_to_mermaid.py generates a Mermaid UML class diagram from a LadybugDB DDL schema file.
python3 scripts/ladybug_schema_to_mermaid.py engine-db/schema/engine-schema.cypher --markdown > schema-diagram.mdOpen schema-diagram.md in VS Code (⌘⇧V) or paste the output into mermaid.live to view the rendered diagram.
make docs writes the repository's own diagrams to engine-db/docs/ and canonical-db/docs/.
scripts/validate_ladybug_schema.py checks a schema by executing
it against a throwaway LadybugDB, so "valid" means what the database actually
accepts rather than what a reimplemented parser believes. Failures are reported
with their source line range and the exact database error.
python3 scripts/validate_ladybug_schema.py engine-db/schema/engine-schema.cypher --verboseNormally reached via make validate-schemas. The statement splitting and error
detection are shared with the loader, in scripts/ladybug_ddl.py.
scripts/wrap_cypher_comments.py refills the full-line //
comments in Cypher DDL to 64 columns, never hyphenating: a word that crosses
the limit stays whole. Only comments change. Trailing comments, /* ... */
blocks, tables, -> paths, aligned columns, list items and NAME = value
lines keep their shape; the docstring lists the rules.
python3 scripts/wrap_cypher_comments.py --check engine-db/schema/engine-schema.cyphermake check-schema-comments shows the diff for both schemas, and
make format-schema-comments rewrites them in place.