Live virtual percussionist for iPadOS, from the same build also on iPhone and on an Apple silicon Mac ("Designed for iPad"). The app listens to an acoustic drummer and plays a shaker that follows tempo and phase — accelerando, rallentando, no loop restart.
Not on Apple Vision, deliberately: the analysis is tuned around a close kit mic and an iPad's own speaker into its own room, and a headset's array is neither. That one is not a build setting — an iPhone/iPad app is offered on Apple Vision Pro and on Apple silicon Macs by default, and the opt-out is in App Store Connect under Pricing and Availability. See docs/PLATFORM.md.
Not a DAW. Not a BPM meter. Not a web app.
Microphone → neural beat / tempo / phase → AUTO lock → adaptive shaker.
JUCE is a submodule, so a plain git clone leaves third_party/JUCE empty and
CMake fails at configure time. Once per checkout:
git submodule update --init --filter=blob:none third_party/JUCE./scripts/setup-ai.sh # ONNX Runtime + Assets/Models/beatnet.onnx
./scripts/run-tests.shWithout AI assets, CMake still builds and TAP tests pass (StubBeatModel). Neural lock needs the setup step.
cmake -B build-host -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build-host --target VPTests
./build-host/VPTests_artefacts/Release/VPTestsVPTests is a console target, but JUCE builds juceaide before anything else
and that one links the GUI modules, so the X11 headers have to be there even
though nothing on this host opens a window:
sudo apt-get install -y libx11-dev libxrandr-dev libxinerama-dev \
libxcursor-dev libxcomposite-dev libfreetype6-dev libfontconfig1-dev \
libasound2-dev
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --target VPTests -j"$(nproc)"
./build/VPTests_artefacts/Release/VPTestsgl, libcurl and webkit2gtk are reported missing at configure time and can
stay missing: no target the tests need links them.
./scripts/setup-ai.sh # ORT host+iOS + beatnet.onnx (skip se già presenti)
./scripts/configure-ios.sh
open build-ios/VirtualPercussionist.xcodeprojIn Xcode: Development Team, pick an iPad or an iPhone, microfono, Run. Stato attuale e limiti del modello: docs/STATUS.md.
Everything in docs/ was measured on an iPad. A phone runs the same engine and the layout is built for it, but no number here has been taken on one.
Simulator (no device signing):
./scripts/build-simulator.sh- USB-C audio interface + kit mic into the iPad, or play a track from the iPad speakers (Spotify, etc.)
- IPAD (default) follows the sound in the room and removes the app's own shaker and congas from the microphone analysis. MIXER is for a close microphone or a line feed.
- Play time — the app is already analysing, but the shaker stays muted until START
- When the state shows FOLLOWING, press START. MIXER enters on the next reliable downbeat; IPAD enters on the next reliable beat because a tablet speaker does not reproduce enough bass for automatic downbeat votes to be trustworthy.
- Speed up / slow down — it should follow without a restart
- STOP mutes the shaker; it keeps listening. START arms it again and waits for the next quantized entry — it does not start at the instant you tap the button. Use TAP when you want to declare the bar's one explicitly.
The app is listening to the room long before anybody plays, and it will find a tempo in an empty room — measured, 99 BPM at a confidence of 0.91 with nobody in front of the microphone. So START arms the shaker but holds it silent until the analysis has heard the input actually start, and the state reads ATTENDO CHE ATTACCHI while it does. Press START early and the part comes in with the band, not before it.
One case it cannot tell apart: a track that was already playing when the app was opened never starts, so it waits. One TAP releases it (so does setting the tempo by hand with FISSO). There is no timeout on purpose: long enough to be a guard is long enough to be a nuisance, and short enough to tolerate brings back a shaker playing to an empty stage.
DINAMICA (next to the styles) is what makes the part sound like somebody listening rather than a loop. With it on, the percussion follows how much the band is giving: it comes down in a verse — quieter and thinner, because a player under an exposed vocal does not play the same figure softer, they play less of it — and in a passage that does not want it at all it stops, at a bar line, and comes back when the band does. While it is standing down the button reads IN ASCOLTO. Off, the part is fixed, which is what some jobs want.
SPOSTA L'1 moves the bar on by one quarter, and the app then leaves it alone. Which of the four quarters is the one is the part of this the analysis is worst at — through the iPad's own speaker it is no better than a coin, measured — so once you have said where the one is, nothing moves it again: not the analysis, not a section change, not a new song. The button lights to say the count is yours. A TAP that declares the one lights it too. Four presses take the one all the way round the bar and back to where it started; the fifth hands the count back to the app and the light goes out.
SETUP (top right) is everything you set once and never touch mid-song: the clock (AUTO, or 44.1 / 48 / 88.2 / 96 kHz), the buffer, MIXER vs IPAD, CASSA, LATENZA, the theme, CLICK TEST and the debug panel — plus a read-out of the rate, buffer, latency and route the hardware actually gave.
Leave the clock on AUTO with a USB interface. AUTO means the interface holds the clock and the app opens at whatever it is already running at, so plugging into an X-Air at 48 kHz costs nothing: no click, and a track already playing through the same route keeps playing.
CASSA is the one worth setting up if you have a desk. Tell it which input carries the kick drum on its own and the app gets the most useful signal a stage can give it: a channel with one instrument on it, which dates the beat to the sample instead of to a 20 ms analysis frame, and which is silent for exactly as long as the drummer is not playing. Measured, that takes the clock's phase error from about 17 ms rms to about 13. The button lights only once the strikes are actually landing on the beat, so naming the wrong input tells you by not lighting.
LATENZA plays a short sweep and listens for it coming back, which measures what this rig really does rather than what the operating system believes about the interface. A second, it works with the band playing over it, and if the send is not routed back it says NIENTE RITORNO rather than inventing a number. Do it once at soundcheck; it is remembered.
CLICK TEST (debug) injects an internal 120 BPM kit so you can verify the engine without drums.
- App status (install / AI / device)
- Architecture
- Neural beat tracking (ONNX)
- Technical decisions
- Roadmap
- Percussioni intelligenti — gap analysis e piano
- Audio engine
- Beat tracking
- Audit del core — tempo, aggancio, salti
- Platform
- Test plan
- Licenses
JUCE is AGPLv3 or commercial. A closed-source App Store build needs a JUCE commercial license.
The percussion recordings in Assets/Percussion/ are from the Versilian Community Sample Library (CC0). See Assets/Percussion/ATTRIBUTION.md.